Tuesday, September 16, 2014

Runtime Extensions

We recently developed an extension model for Voxel Studio and Voxel Farm in general. The idea is you should be able to come up with entirely custom procedural content without having to recompile any of the tools, and even your final EXE if you like.

You can achieve this by wrapping your custom code inside an extension. During world design time, Voxel Studio is able to load your extension and allows you to input whatever configuration parameters you have chosen for it. Then, during runtime, the same extension code runs thus guaranteeing you get the same results you saw inside Voxel Studio.

Let's follow a quick example. Imagine we have developed a small scene in Voxel Studio using the default terrain component. At this point we have interacted only with the vanilla settings, so our scene looks like this:


Note this is only terrain, it does not contain other layers like rock and tree instancing, but it should be enough for this example.

Now let's say we want to add a massive sphere somewhere in the image. While we could go in edit mode and add a sphere using a voxel brush, this would set a lot of voxels. Since we know this will be a perfect sphere we can save a lot of data if we just store the center and radius and produce the voxels on the fly. Voxel Studio does not include a layer like that out of the box, but we can create it ourselves. Here is how:

Voxel Studio in Windows OS can load extension DLLs at runtime. You can develop the DLL in any form you like as long as the few required entry-points are found. The first few are functions so Voxel Studio and Voxel Farm in general can ask the extension what parameters it wants to capture. And then there is one function that will return the voxel data for a given chunk of space.

So we create a new DLL project. Just by dumping the binary DLL in the extension folder, Voxel Studio should be able to find it and allow us to use it for a new voxel layer:


Here our extension has identified itself as "Mega Sphere". Clicking on it will add it to the list of voxel layers in the tree.

We then define four properties for the sphere: origin x, origin y, origin z and radius. Exposing property metadata is what allows Voxel Studio to create editors for the extension without really knowing what they are and how they will be used:


Now comes the real work. So far it was mostly about metadata, let's see how we get actual voxels out of the extension. It comes down to implementing a function that looks like this:

        bool getVoxels(
                char* object,
                VoxelFarm::CellId cell,
                double cellOrigin[3],
                double cellSize,
                double voxelSize,
                int blockSize,
                VoxelFarm::VoxelType* changeFlags,
                VoxelFarm::MaterialId* materials,
                VoxelFarm::Algebra::Vector* vectors,
                bool& empty)


I will not go into the implementation this time, but the overall idea is this function is expected to fill the material, vector and flag 3D buffers with voxel data. The requested region of space is defined by cellOrigin and cellSize.

Once we code this to output a sphere, we are finally able to refresh our render view and see the results:



Here you can see some spheres. The one in the last image has a 10 km radius. Naturally we could have developed a more interesting layer, for instance a meteorite impact zone or ore veins, but hopefully you get the idea.

One last thing: Using native code for extensions always brings up the issue of security. We debated this for a while when designing the system. We finally chose to use DLLs just because they allow  to run native code without penalty. You can get really close to the metal. The security aspect can be managed by certification, also by running the DLL in a lower OS integrity mode, thus restricting the kind of access it would have over the system. And of course you can always have a DLL extension you trust that acts as a wrapper for code you do not trust, but runs in Lua or some other form of application language where you are certain it can be contained.

15 comments:

  1. Just being curious, what UI framework are you using for Voxel Studio?
    Your work is amazing!

    ReplyDelete
    Replies
    1. For the UI we use DevExpress. It is quite good and clean. The UI is a managed C# application, but some of the tabs runs in separate processes, so a big part of the code is native C++.

      Delete
  2. So do you see this as a method of pushing new content to games, as method to allow fewer build cycles while continuously updating content, a method for mod support, or a primary way to help condense terabytes of data down under a gig?

    ReplyDelete
    Replies
    1. All of the above.

      The initial goal was to allow modding from the start, both at design time and runtime. Now I am seeing this is great for prototyping new voxel layers as well.

      Delete
  3. Sorry about the off-topic.
    But what is your opinion of new voxel friendly features from Nvidia Maxwell, if you have had time to look at them. DX11.3, Volumen Tiled Resources and Conservative Rasterization. And about Voxel Global Illumination, it could be tricked to acelerate Voxel Studio?
    Thanks.

    ReplyDelete
    Replies
    1. It is great to see hardware makers talking about voxels.

      It looks some of the tough remaining problems in realtime graphics like global illumination and occlusion culling will finally be solved by voxels, even for standard polygon based content.

      What I would really like is for them to create a standard voxel format (maybe license Voxel Farm? :P) and give us hardware voxelization and contouring. That is, hardware conversion from poly to voxel and from voxel to poly. We can take it from there.

      Delete
    2. I don't think Nvidia Maxwell has any specific voxel friendly features. It's just a faster card that can do Voxel Global Illumination at acceptable frame rates. It is also going to be very expensive initially.

      Delete
    3. Aren't the first Maxwell cards (gtx980,970) actually cheaper than the outgoing cards while performing similarly?

      Delete
  4. I appreciate your writing because you described really well. Thanks for sharing such an informative post.

    ReplyDelete
  5. question : linux support ?

    ReplyDelete
    Replies
    1. The engine is standard C++, you should have no problem building and running in Linux. This tool UI however is built using .NET. It is mostly a front that talks to a native process that runs the real engine.

      Delete
  6. Hi Miguel, may I ask you what kind of GUI framework do you use for Voxel Farm ? It looks very slick indeed ! :-) Thanks !

    ReplyDelete
    Replies
    1. We use DevExpress for docking panels, menus, and pretty much everything in the UI.

      Delete
  7. Is there any documentation for making an extension?

    ReplyDelete
    Replies
    1. Yes, you can find several topics on extension here, including full code for an example extension:

      http://www.voxelfarm.com/doc.html?creating-an-extension

      The community has also created some interesting examples. Here is one great extension created by Namrog84, source code included in the post:

      https://forums.voxelfarm.com/post/basic-cube-world-with-underground-and-rudimentary-ore-vein-prototype-9631187?pid=1302998265

      Delete