«
»

Hi. Name's Abdi Dahir and this is a place for me to mark the web in w/e strange way I feel.

I have a ton of interests and maybe not enough focus. I will have a little more to show for myself then a fractal patterned orb soon.

1K3tSx6hgz13Vx4A8Rt6pKZWNnBSFLEu3f

on the ground

  • tdot: bahen center

  • brooklyn: outpost

  • sf: another cafe

  • sf: social study

  • vancouver: nemsis

  • work: relic

Dec 17 2017:

Improved engine and new flames

It's been a crazy year, I've moved to Vancouver, jumped jobs twice and am more deadset then ever on building this thing out

I've learned alot at artillery and maxis, mostly about engine design and entity component systems. I've trashed most of my old code and rebuilt the voxel engine as a proper game engine. I've put in alot of work on the lua bindings and now have a clean interface between lua code and c++ code that's all handled behind the scenes my the engine. All C++ components and events can be have lua code bound to them by simply adding handlers such as OnEventName(arg1, ... argN) with no fancy bridge code other then a single macro, DEFINE_EVENT(onEventName, argType1 ... argTypeN). Theres also macros for exposing any memeber variable of a class to lua allowing for the lua file to both read and set the value. Its cool. I'll show it off here at somepoint.

Thats all nice and everything but I still haven't built a new game since 2014 and I want that to change this coming year. I'm working fast towards a prototype trying to minimize distractions (Its so hard to not work on fun rendering stuff). My current fulltime job has great work life balance so things are looking up.

You will hear from me again soon!

April 30 2016:

UI and text rendering woes

Work work work, but I've made some time to look at the engine a bit more.

So I've been looking at performance and started measuring specific parts of the rendering pipeline. I haven't been getting good FPS lately and I mostly assumed it was due to the voxel code, hence all the threading work. What I forgot to pay attention to was the text rendering code I slapped in! It alone took up two thirds of my rendering time! It obviously needed to go!

I've been using FreeType following this very helpful guide which got me to a point where I could simply write text to the screen. However due to the lack of caching and constant texture generation, my FPS took a serious hit. At the time, I was still at 60FPS (I normally have vsync on blah) so it went unnoticed and performance was not my first priority.

I had the option of figuring out how to improve on this or move away from it entirely and finally start looking at some UI libs. I decided to go with the latter and have started using ImGui a very self contained library that does basically everything I need currently. My FPS on my windows machine is now at something ridiculous like 1200FPS in top down view and around 500FPS in free cam.


You can view the UI Demo here.

Obviously I'm pretty happy with the switch, I'm not too sure how practical this lib will be in the long run but I'm tempted to find out :p

January 3 2016:

Persistent Data && Threading Opts

So as you may of noticed, I took a bit of a break due to the new job. Work is exciting and consuming 110% of my time but since I've been on break, I obviously took the time to tackle more of the game.

So although I did do most of the multithreading work in July, I did not take the time to debug it thourghly or truly break down all work to be exclusively local to chunks. i.e lighting was not threaded. More importantly, rendering required all threads to stop before I could render out a chunk. I ran the risk of corrupting data being bound to the gpu. Fortunately I banged my head against my keyboard to sort that out using the minimal amount of blocking code. Threads no longer need to be barrier synchornized while I write to GPU memory.

The solution mostly boiled to keeping two states, the currently bound state of the chunk, and the current active state of the chunk. I maintained seperate values for things such as block count, x,y,z regions start and length in the buffer etc. One for the version that is currently bound to gpu memory and one for the version that is not. This does not mean chunk data was duplicated 100% of the time, it just means the vertex data thats normally just on the gpu and sometimes on the stack was kept around longer then before and was only cleared at the time of the next glBindBuffer call. When that call is made I copy the current state synchronisly (just few integers needed) and release locks allowing any thread to attempt to reupdate the vertex data if need be. This allows me to render and recreate meshes at the same time.

Lighting basically required a rewrite of the floodfill algorithm which I originally got from the guys making Seed of Andromeda. The change mostly revolved around the addition of a lock and local queues.


I made a quick demo earlier to demonstrate that portion of the update.

After feeling pretty confident about my threading skills I decided to crush that confidence by attempting to thread reading/writing chunks. I did manage to get this all working but my self-esteem did not hold out unfortunately. Currently I am writing out each chunk to its own file based on it's position. Since I reuse chunks as I move around the world, assigning a chunk an id and using that as the file name is not helpful. Position's worked fine as long as I made sure to synchronize read and writes to a chunk file. Otherwise I run the risk of overwriting file with chunk data of chunk that was recently reassigned the conflict creating position.

The synchronization between the update calls and the read/write calls was mostly a freebie due to how I organized chunks that required unloading. If a chunk is in the unloading queue, it cannot be repositioned or put into the update queue. Only chunks stored in a free queue can be reinited.

Thanks to SDL's SDL_GetPrefPath(orgname, dirname) function, there was very little os specific code necessary. That call provided the correct directory to write to, the only extra work was making a safe mkdir call which I did like so.


int mkdir_safe(const char* path) {
#ifdef __linux__
  return mkdir(path, S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH);
#else
  return mkdir(path);
#endif
}


Please excuse the laziness, ideally the #ifdef would be defined for windows and default to the posix mkdir call, however I did not want to enumerate the possible windows/windows compiler flags

Finally a point on the format, currently i'm writing out the chunk byte for byte, so each chunk file is always 32KB. I intend to run encode the files shortly. The next step will be to organize multiple chunks into a single file, ala minecrafts region files. I do not envision that being too complicated given the right amount of synchronization.

BUT OMG IT WORKS!!!

Yea persistence, woo! The next thing I'd like to tackle is building out an entity/component system. Work has inspired me to build out a proper game engine. Having a clear separation between engine and game code at work allows everyone to make a tremendous amount of change on the fly. I really want that. I will have it!

July 29 2015:

Cross compilation with mingw

So first, I'd like start out by thanking all the compiler people out there. Honestly, you don't get enough love. You're all the best.

My experience with emscripten has been mostly great, ignoring the SDL2 hiccups I ran into, porting to the web was surprisingly easy. I feared the worst for mingw though, so I originally attempted to port the code over to Visual Studio. ... which was incredibly painful.

Turns out mingw is not so bad, as long as you are on Arch Linux! Every library that you would like cross compiled has been packaged up nicely for you in the AUR. The only thing you need to do is include them. I did this and boom my code ran on ...


Winnnnddoowwws.

The only tricky bit is sorting out the Makefile required. I've attached the one I used which will hook up your SDL2, GLEW, freetype GLM and lua dependancies. I installed each one of this packages again using the AUR mingw-w64-* version. I believe 64 bit is required to get C++ threading to work out of the box. Either way, you can pick to build a 32 bit version simply by changing the architecture in the Makefile from x86_64 to say i686.


# Build tool
CC = /usr/bin/x86_64-w64-mingw32-gcc
CXX = /usr/bin/x86_64-w64-mingw32-g++
# Build flags
CPPFLAGS = -O3 -std=c++11 -Wall -pthread -DGLEW_STATIC
# Includes
CPPFLAGS += -I/usr/x86_64-w64-mingw32/include/ -I/usr/x86_64-w64-mingw32/include/freetype2/
# LD Flags
LDFLAGS = -L/usr/x86_64-w64-mingw32/lib
# LD Libs
LDLIBS = -static `/usr/bin/x86_64-w64-mingw32-sdl2-config --libs` -ldinput8 -ldxguid -ldxerr8 -luser32 -lgdi32 -lwinmm -limm32 -lole32 -loleaut32 -lshell32 -lversion -luuid -lglew32 -lopengl32 -lm -llua -lfreetype
# Source
main: yourcode.o
# Build Souurce
all: main


I'm a bit of a Makefile noob, so please excuse any strangeness. (Like how i'm creating a 'exe' main without the proper extension

A few interesting things about this Makefile.
1. The mingw32/include/ section deals with GLM and Lua automatically as I installed both of them via AUR so they were added into the folder correctly. Even though GLM doesn't need to compiled, Its more convient to just have it included the same way the other libs are
2. -DGLEW_STATIC and -static flag are REQUIRED, mingw is more then happy to dynamically link to libraries, this forces SDL2 and GLEW to compile statically
3. I have no idea what the libs after the sdl config to -lglew32 are, but I do know that SDL2 will not link without them. They seem to have something to do with window's keyboard handling?

I also put in a TOOOON of working making the terrain infinite. I would love to dedicate a blog post to it but I don't think I'm done with it just yet. I would love for it to be faster, but hey ... looks pretty no?

Really though, I intend to make the game ... soon. Next post HAS to be update game content.

July 16 2015:

Multithreading in OpenGL

So whenever I read posts by voxel engine writters that list features they've implemented, Multithreading is almost always listed as a simple bullet point. They then go on to talk about meshing problems, LOD etc. There's very little discussion on how they implented it. I'm gonna try to change that cause damn does it ever suck to multithread code with OpenGL being single thread only.

So I lied a bit, OpenGL does support multithreading with shared contexts but as far as I can tell, the API to do it is windowing system specific, i.e OS specific. This wasn't really a path I wanted to go down. Thankfully, its not that big of deal, it just lead to a very important condition that I needed to mantain.

ALL glXXXXX Methods must happen on thread that inited the context!

This was a bit unfortunate, I was originally planning on threading my mesh generation code directly, i,e no changes to its implementation. It originally created a fixed size buffer on the stack and stored all the mesh vertecies. This was then immediately bound to a chunks VBO and discarded when the function returned. The last step, binding to the VBO would not be possible anymore if this were running in a seperate thread. This ment that the mesh could not be stored on the stack, which then lead to the abstraction of the mesh and heap allocation for its storage. The code was updated so that the chunk had a reference to a mesh object, it would write to it when the mesh generation function was called and cleared after the main thread bound its data to a VBO.

Now in order to define what work a thread would do and what the thread would do after its completion. I created a threadPool class with two queues. A workQueue and a responseQueue. The workQueue would take a task structure which contained data to be worked on and a C++ lambda function (yea! lambdas!) to execute on this data. The responseQueue would take the result of the lambda function. The two queues would be synchronized with their own mutex and wait channel. I'm actually very proud of this threadPool class, I'll be posting it on github soon, its just a header and it provides basically everything I need for controlling the threadPools work flow, pausing work, adding work etc.

So when it comes time to render the scene, I pause all active threads and I drain the responseQueue of its completed meshes, binding them all. I compute visible chunks and for any chunks that need updating, I push it on to the queue with a lambda function executing its mesh generation method. After the scene is rendered I resume all threads! Boom multithreading!

... Except for the work order. So if you were to just keep pushing chunks that needed updating onto the queue, your worker threads would not be able to keep up, causing them to continue to work on chunks that arent visible to the player anymore. You could use a stack instead of a queue to deal with this issue. On top of work order, at least for me, I had issues where the same chunk was on the workQueue multiple times. I do not do any checking to see if the chunk exists in the queue or if it still needs updating. I prefered to just wipe the workQueue during the pause and drain response phase and mantain the queues FIFO ordering instead.

After getting all that sorted out, I gained level of detail for free. I can set the 'sample' rate in my mesh generator (It just sets the increment step in the various for loops) based on distance away from the player and push the chunk on to the workQueue if the sample rate changed.

Of course, this leads to issues in areas where chunks at different LOD meet. I got around this by extending each chunks mesh by one sample step. It works pretty well, there's still times where holes are visible but I enjoy the effect enough to not care.


Here's a vid of it.

I swear I'll get back to the game part of this reaaalll soon.

July 10 2015:

Voxel Segmentation

I've been working on how enemies should be fought in my game and I've decided that in order to best take advantage of the engine, enemies should die by being broken apart. i.e whenever an enemies body becomes disjoint, all or one disjoint set of their body should be destroyed. Sounds fun buuuut its been a pain getting in order. This post will discuss what I've learned about set segmentation for voxel data.

First my assumptions:
1. Enemy starts off completely connected
2. Enemy loses at most one voxel at a time
3. Diagonal voxels are not considered connected

Nothing crazy but should be stated. With this I began building out a soloution. I initially came to the conclusion that the only way a set could ever become disjoint is if the voxel currently being removed was one of only three voxels in a neighbourhood around the voxel. The idea was, if we assume up untill this point that the enemy was fully connected (i.e there exists a curve in between any two points on the enemy such that every point on this curve is contained in a voxel of the enemy) then removing a single voxel is only destructive if it was the only point between it and its two neighbours!

So now the problem became this: If a voxel that just got removed fit the above description, it being one of three voxels in a neighbourhood. How do I know if it created a disjoint set? How do I know that there doesn't exist another path between the two voxels neighbouring this guy that I just orphaned?

Does there exist another path between these two voxels?

Sounds like a path finding problem, sounds like I'm writing A*.

So I did this.
Attached is a video of it in action. After determining if removing a voxel creates two disjoints sets. I flood fill the two sets using the orphaned points as seeds and paint them red or blue. Since my player character deletes multiple voxels at a time, when a disjoint set happens to be small, it may be deleted immediately by the characters missile. Hence the case where the enemy is painted fully blue or red.

Seems to work right? WRONG!

You may of guessed this but the assumption of there being only two orphaned neighbours possible is based on the assumption that removing a voxel can create at most two disjoint sets. This is incorrect. At most 6 orphaned neighbours are possible creating 6 disjoint sets IF the removed voxel was say the center of some star like enemy. Imagine 1x1 path shooting out of every face of this to-be-removed voxel. The set was originally connected, however this one little piece was holding it all together. A* on just two voxels is not enough ...

I'll post an update when I sort this out.
In other news I have lua bindings now wooot :p, ill do an update on that too!

June 28 2015:

It begins...

So I've started working on my third attempt to make a game out of my voxel engine. I believe in this one alot so I'm going to start documenting some of its developement.
I'm making a voxel space shooter crossed with megaman. Think megaman bosses but in space and with destructiable bodies.

Destructiable enviornments are pretty straightforward in a voxel engine, the volume data for an enviornment is static so intersections are pretty trivial. Chunks are generally axis alligned too, meaning no funky transformations are needed to check point collisions. My engine currently organizes fixed sized chunks in a octree, so when I want to shoot one of my asteriods I query the octree picking up chunks along the ray as I pass em and tracing through them to look for collisions.

But what happens when these chunks are rotating around and moving? For my voxelized enemies this will be pretty common so my first task was to sort out intersections with arbitrary positioned/alligned chunks. My current soloution, like most raytracing renders (mine is not fyi) is to transform any ray into the chunks object space. This can be done my multiplying the incoming ray by the inverse of that chunks model matrix (its translations, rotations etc.) and checking to see if the ray crosses then. Its a simple trick and solves my problem.

I use this trick as well to determine which faces of the enemy to render. I transform the cameras position into the enemy chunks object space which allows me to simply check to see if the camera is infront of or behind a voxel face using the extents of the chunk. This would not of been possible in world space where a face could of been unalligned with the axes.


I've uploaded a demo video to show how it works.

Hopefully I'll have more to show you in the future.