Details

Category: Blog

Over the past few years with regard to virtual worlds I've found myself in a number of discussions regarding which platform is more "stable" or "secure".    I did find myself in a position to make some comparisons when we were deciding what software to use when setting up Utopia Skye Grid.  I was of course very familiar with Halcyon, having made a number of substantive contributions to it over the past few years.   There was OpenSim as well and it had progressed since Halcyon was forked.  It too deserved a look.

As many are probably aware, Utopia Skye Grid is running OpenSim 0.9.  So the fact that we arrived at this being the best platform to move forward with should tell you something.  But I'd like to expose my thought process in getting there since there has been alot of FUD (fear, uncertainty and doubt) and misinformation regarding both security and stability with both platforms.  I will say there's no out of the box production ready solution and settling on OpenSim for our grid brings with it a pile of tasks I'll undertake to get things to the point I feel they need to be to really be "stable" and "secure".   We're in Beta now in part for that reason.  But this is the path we're on and I feel it's the right decision.

I had 2 key requirements when looking at technology for Utopia Skye.  1) the environment had to be verifiable and testable in a way that I could quickly tell if there was a regression when making changes.  The coding on USG is largely going to be me so I need to know if I broke something fairly quickly.  And 2) We need an audience.  There has to be a way to bring people into the environment, either as residents or via a linking mechanism like the HyperGrid.

Regarding code quality experience has certainly shown that the quality of the code in Halcyon has dropped over the past few years.  There were a number of unresolved issues with asset fetching that plauged InWorldz for at least a year or more prior to it's shutdown.  Also teleport and crossing issues.  And many will remember regressions with sit targets being lost because software was deployed without, IMO, adequate testing.   Related to that if you look at the unit test cases in both the OpenSim and Halcyon you get a pretty telling story.  In Halcyon there are roughly 60 unit tests for the entire platform and of those about half failed at last examination.  Of those tests the recent ones added were introduced by myself or Tranq/David when he was active with the project.  Contrast that with OpenSim DEV.  There are over 600 tests all of which run and pass that provide test coverage over most of the existing codebase.  I'm able to run this test suite after every compile and verify if there has been a regression in the code with a change I have introduced.  This is in fact my current test and release procedure.  It's not perfect either but there is a night and day difference with the level of support and reliability that you get from the test suite in OpenSim vs. what is available in Halcyon.

I alluded to this next point above.  Regarding quality and testing many experienced the impact of that with regard to Halcyon on InWorldz.  This next point isn't related to technical quality (though you could say that one technical problem; the asset fetch times; improved when all the data was lost) but ultimately it's hard to describe a platform as secure or stable when it can simply go away and take all of the residents data and financial investments with it.  This is in part due to the closed nature of the grid and since it was often argued that made things more secure I think it's valid to discuss it and indicate it was in fact a reason I took the decision I did away from Halcyon and towards OpenSim.  On a Hypergrid connected grid, if the grid owner indicates they will shut down you have a way to potentially move and save some data.  On a Hypergrid connected grid that isn't running its own valueless currency the tokens I have can still have value if they are hosted on a service that's useable across more than one grid (Gloebits for example).  In short the lack of lock in with OpenSim on the Hypergrid is itself a benefit and a factor in how secure and stable the overall solution is.  Everyone that lost what we invested years building in InWorldz is painfully aware of the truth of that.

Secondly we needed an audience for our GT shows.  A closed grid meant we'd need to build up all local residents for the shows.  And while I think what we're trying to do with USG is compelling enough it's VERY difficult to bootstrap a closed grid at this point.  We don't have the benefit of reusing most of the data from all the work we did for Golden Touch for InWorldz.  That data is gone with the grid it was hosted on (and IMO the attempts to bootstrap Islandz with content from OAR's from IW is not only technically unfeasible it's certainly unethical and violates the contract terms of InWorldz own TOS).  The data simply doesn't belong to Islandz.  It belongs to the content creators and users, all of which lost it when InWorldz "went away".   There has been some discussion about doing a "more secure Hypergrid around Halcyon". This is mostly hearsay on my part because for a number of reasons I'm unable to participate in the discussions that take place around Halcyon and Islandz or to represent my own work on it.  But I can say with absolute certainty all the Hypergrid support that was in Halcyon was ripped out some time ago.   So you'd need to implement such support from scratch.  IMO there isn't sufficient development talent or resources being applied to Halcyon to pull off a project like that.  So because I had a clear requirement for Hypergrid access in order to get people into the grid for our Golden Touch shows as well as participate in our community Halcyon was clearly a non-starter.

Halcyon due to simple lack of technical attention and care and feeding has fallen behind.  That will sound shocking to some.  I think you can trace some of that to a few key contributors backing away from the project, most specifically Tranq who really provided the direction and cohesive glue that kept the project moving in a positive direction. There was truly some innovative work done over the years he provided leadership.  Unfortunately there was also a fair bit of code rot, improper implementation and breakage introduced after he "left".   My own contributions largely tailed off during this period in large part due to frustration with how the project was being maintained and simply feeling like testing was being done at the expense of the residents in IW.  We all experienced the many ways that Halcyon for IW was broken and attempts to address those things via technology changes were often done against the advice and feedback of a number of senior contributors and even Tranq himself.  It does beg the question, why not go back and fork it from the place where Tranq/David stopped contributing.  This is possible but doesn't address my comments about the need for functional Hypergrid access in order to provide a way into the environment.  IMO Halcyon is most valuable as a repository and proof of concept of some key technolgies developed over its life.  As a platform for hosting a virtual world it fails on a number of points that I've discussed above. For that reason I rejected it.

So, OpenSim.  There are issues with memory leaks and occasional lock ups.  Its a big, complex piece of software.  In some cases overly complex.   Some of the problem spots are things that we fixed over time in Halcyon and so there is a blueprint for how to do it here as well.  That supports I think my statement that Halcyon as a technology repository still has value.  And there are some specific changes we did (handling async http, Mesh land impact, Phlox for scripting) that could and should be ported over to add value to OpenSim.    There has been a similar argument to Halcyon that there are destabilizing changes in 0.9, and work should be done from a 0.8.x tree to move OpenSim forward.  I think that there is some truth that some of the work that went into 0.9 has problems but in my opinion the overal quality and stability of 0.9 is good. Enough so that moving forward with it and fixing the issues is the appropriate approach.  There is the open question of how healthy the actual OpenSim project and governance model is as well. I believe that needs some attention but that is a topic for another post in the future.

I hope you've found this interesting.  My goal with Utopia Skye Grid is to provide a platform for us to continue to do our Golden Touch dance shows, build up the community of creators via the Creators Guild and generally create a community around our use of OpenSim that builds each other up and supports our work.  In line with that we'll improve OpenSim and make contributions available as I can so that we ultimately end up with the most stable, secure virtual world platform we possibly can.  I welcome you to join us in that effort. 

Mike