14 Apr 2009

New N3 SDK (Apr 2009)

The new SDK release should be ready by tomorrow (unless there's some last-minute fuckup). For now here's a rough What's New list since the last SDK from September 2008:

Tools:
  • new command line tool: archiver3 - multiplatform wrapper for generating file archives
  • new command line tool: n2converter3 - convert .n2 files to .n3 files (Nebula graphics objects)
  • new command line tool: suiconverter3 - batch converter for SUI resources (simple user interface), currently only useful for the Wii port
  • new command line tool: synctool3 - sync local project directory from build server (only useful with our inhouse asset pipeline)
  • new command line tool: countlines3 - count N3 source code lines and generate Excel-compatible csv-file (comma separated values)
Foundation Layer:
  • brought Wii port uptodate (not part of the public SDK of course)
  • Util::CmdLineArgs renamed to Util::CommandLineArgs
  • Scripting subsystem moved into an addon
  • Remote subsystem moved into an addon
  • new Macros __ConstructInterfaceSingleton / __DestructInterfaceSingleton
  • new standard define __MAYA__ if compiled as part of a Maya plugin
  • new concept: ExitHandlers (see Core::ExitHandler)
  • new low level debug feedback method Core::SysFunc::MessageBox()
  • new concept "root directory" (see CoreServer::SetRootDirectory)
  • various changes to enable using N3 code in N2/Mangalore apps (N2 now sits on top of a very slim N3 Foundation layer)
  • SizeT and IndexT now signed (had to be done for N2/Mangalore compatibility unfortunately)
  • IO::Console is now an InterfaceSingleton
  • various debug HTML page handlers can now sort table content by columns
  • Debug::DebugServer is now an InterfaceSingleton
  • added a minimal Debug::HelloWorldRequestHandler as example for a simple HttpRequestHandler
  • new comfort method Http::HtmlPageWriter::TableRow2(), saves code when creating a 2-column HTML table
  • added Http::HttpClient class, allows send requests to HTTP servers
  • Http::HttpInterface is now an InterfaceSingleton and derived from Interface::InterfaceBase
  • new class Http::HttpRequestWriter
  • new class Http::HttpResponseReader
  • added a "single thread mode" to HttpServer
  • added new classes InterfaceBase and InterfaceHandlerBase
  • moved IOInterface-stuff under IO
  • new class Debug::ConsolePageHandler, displays console output in web server
  • generalized Zip filesystem stuff into general archive filesystem, with ZIP support as one specialization (on Wii, ARC files are used instead)
  • ZipFileSystem is now an InterfaceSingleton (no more per-thread wasted memory for archive table-of-contents)
  • added an "AsString()" method to IO::FileTime
  • new methods in Win360FSWrapper: SetFileWriteTime(), GetAppDataDirectory(), GetProgramsDirectory()
  • moved IO::ZipFileStream class to io/zipfs
  • moved path assign methods from IoServer into new InterfaceSingleton: IO::AssignRegistry
  • new standard assigns under Windows: "appdata" and "programs", mainly useful for tools
  • bugfix in IO::ExcelXmlReader for tables with empty cells
  • new class IO::HistoryConsoleHandler, captures console output into a ring buffer (used by Debug::ConsolePageHandler)
  • moved URI scheme methods from IoServer into IO::SchemeRegistry InterfaceSingleton
  • removed critical section from Stream::Open / Stream::Close
  • new method: IO::XmlWriter::WriteComment()
  • new methods specialized float4-loading methods: Math::float4::load_float3(), Math::float4::load_ubyte4n_signed()
  • vector comparison methods in Math::float4 more intuitive and flexible
  • Math::matrix44() default constructor now sets object to the identity matrix (default constructor still empty in Math::float4!)
  • new constructor from float4 in Math::quaternion
  • moved lots of math functions from scalar.h to platform-specific d3dx9_scalar.h to enable platform-specific optimizations
  • Memory::Heap constructor now accepts initial and maximum heap size
  • memory leap detection for Memory::Heap (doesn't quite work as expected yet)
  • removed global heaps: SmallBlockHeap, LargeBlockHeap, StringHeap
  • added global heaps: PhysicsHeap, AppHeap, StringObjectHeap, StringDataHeap
  • new experimental Win360MemoryPoolClass
  • Messaging::AsyncPort: can now add message handlers to already opened ports
  • added "deferred handled" flag to Messaging::Message (used in the rendering thread to keep CreateEntity messages around until their resources are loaded)
  • wrapped socket and TCP/IP classes to enable "TCP/IP-over-HIO2-Tunneling" on the Wii
  • System::Win32Registry(): split Read() method into ReadString() and ReadInt()
  • System::ByteOrder::Convert<TYPE>() methods now return a value, added new methods System::ByteOrder::ConvertInPlace<TYPE>()
  • new methods Threading::Interlocked::Exchange() and CompareExchange()
  • Win360Thread::IsRunning() now uses the Win32 function GetExitCodeThread() to detect whether the thread is running
  • new method Threading::Thread::YieldThread() (gives up time-slice)
  • new class Threading::SafeFlag
  • new concept: Timing::MasterTime and Timing::SlaveTime, main thread has a MasterTime object, and distributes "main time" to slave threads (i.e. the render thread)
  • Util::Array now has a MinGrowSize and MaxGrowSize to prevent excessive memory waste for huge arrays
  • new experimental class Util::Delegate (doesn't work in Codewarrior unfortunately)
  • new method Util::FixedArray::AsArray()
  • new methods Util::Dictionary::KeysAs<TYPE>(), and Util::Dictionary::ValuesAs<TYPE>()
  • new class Util::PriorityArray (which actually isn't needed anymore I think)
  • new class Util::RandomNumberTable
  • new class Util::Round
  • made many non-essential Util::String methods non-inline
Render Layer
  • new Character subsystem (modular character rendering, fresh rewrite of N2's Character3 system)
  • new CoreFX subsystem (ported from Mangalore's VFX subsystem (visual effects)) -> NOTE: will be moved into addon
  • new CoreUI and UI subsystems (simple user interface system) -> NOTE: will be moved into addon
  • new Video subsystem (video playback, currently Xbox360 only) -> NOTE: will be moved into addon
  • new Particles subsystem (rewritten from scratch) -> NOTE: will be moved into addon
  • new PostEffect subsystem (ported from Mangalore) -> NOTE: will be moved into addon
  • new Vibration subsystem (game pad vibration support) -> NOTE: will be moved into addon
  • new Vegetation subsystem (Drakensang's grass renderer, currently broken under N3) -> NOTE: will be moved into addon
  • new concept: RenderModules, clean framework to add functionality to the render thread
  • new concept: AnimEvents, animations can emit events at certain sample times (i.e. for playing foot-step sounds at the right time)
  • new concept: character attachments (swords, etc...)
  • lots of bugfixes and improvements to Animation system
  • new concept: AnimDrivenMotion, synchronize character movement with its current animation
  • new concept: batched messages, drastically reduce communication overhead between threads by client-side batching of messages
  • new methods: Audio::AudioEmitter::Pause() and Resume()
  • new methods: AudioDevice::SetGlobalVariable() / GetGlobalVariable()
  • added/fixed Xbox360 support to XACT audio classes
  • new class: MultipleRenderTarget, wraps MRT rendering
  • new class: MouseRenderDevice (currently only implemented on Wii)
  • added support for GPU-instanced-rendering
  • RenderTarget: added support to resolve depth buffer into texture (Xbox360 only)
  • RenderTarget: added support to create a resolve-texture which can be efficiently read by the CPU
  • added "late-binding" to ShaderVariableInstance
  • D3D9StreamTextureLoader and D3D9Texture moved from win360 to d3d9, since specialized Xbox360 versions exist now
  • Debug::MeshPageHandler can now display a dump of the vertex data in the web browser
  • Debug::TexturePageHandler now displays the current resource state of textures (Initial, Pending, Loaded, etc...)
  • new class CoreGraphics::MemoryMeshLoader
  • renamed CoreGraphics::Shape to CoreGraphics::RenderShape (because of CodeWarrior problems with identical filenames in different directories)
  • added Multiple Render Target support to Frame::FramePass and Frame::FramePostEffect
  • Graphics::Display::GetDisplayMode() now returns display mode actually set by the CoreGraphics::DisplayDevice (may differ from the requested display mode)
  • Graphics::GlobalLightEntity: all light parameter changes are now transferred to the render-thread-side post-creation
  • Graphics::GraphicsEntity: internal entity handle now only becomes valid after resources have been loaded on the render-thread-side
  • Graphics::GraphicsInterface now uses batch messaging to communicate with render-thread (only 1 message sent per frame)
  • Graphics::Handle is now a smart pointer (fixes problems where render-thread graphics entities were disposed too early)
  • Graphics::ModelEntity: support for AnimDrivenMotion
  • new Input::GamePad methods: ButtonAsString(), AxisAsString(), GetStateAsInputEvents()
  • InternalGraphicsEntities are now registered with the InternalGraphicsServer
  • lots of changes in InternalGraphicsServer and InternalModelEntity which require a proper cleanup doh
  • added support for 2-sided lighting to global lights
  • new classes: Models::AnimatorNode and Models::AnimatorNodeInstance (legacy Nebula2 stuff)
  • new classes: Models::CharacterNode and Models::CharacterNodeInstance, integrate character rendering with model nodes
  • loading of ModelNodes completely rewritten (new .n3 file format, plus n2converter3 tool to convert .n2 files to .n3)
  • new Model::OnResourcesLoaded() method, if Model subclasses need to do initialization work after resources have finished loading
  • new methods to lookup ModelNodeInstances on ModelInstances
  • new class Models::StreamModelLoader
  • lots of other minor changes in namespace Models
  • new class: RenderUtil::MouseRayUtil, convert 2D mouse position into world-space 3D-ray
  • new method: ResourceManager::CheckPendingResources(), returns true when there are currently no pending resources waiting to be loaded
  • SharedResourceServer: several methods now accept a ResourceLoader object when creating shared resources

I also did some interesting line counting statistics recently for the Foundation and Render layers (hopefully the images won't be scaled down too much):

foundation_layer 

render_layer 

"General" is the platform-agnostic code, which is the same for all platforms, "Win360" is the code which is identical between Win32 and Xbox360.