OpenSim in Visual Studio on Win64

Note! Article applies to developers using Visual Studio on 64-bit Windows.

Visual Studio is a great tool. Most developers who can use it should.

Currently OpenSim has some problems with 64-bit mode. This is because we have some native .dll's and .so's such as:

• SQlite
• ODE
• OpenJpeg
• + more

SQLite you can do without by choosing MySQL or similar. But the rest is a bit more difficult.
See http://www.adamfrisby.com/blog/2008/08/running-opensim-under-a-64-bit-environment/ for more info.

To solve this problem we have a file named "OpenSim.32BitLaunch.exe" which is compiled with CPU target set to 32-bit. This in term loads OpenSim.exe as a standard module and executes it.

But this is only from command line. If you want all the glory of debugging that Visual Studio provides then this workaround will work:

1. Download the OpenSim source as usual, run prebuild as usual.
2. Open OpenSim.sln in Visual Studio.
3. Right click solution and choose Add->Existing Project...
4. Browse to "OpenSim\Tools\OpenSim.32BitLaunch" and add the project "OpenSim.32BitLaunch".
5. Right click project called "OpenSim.32BitLaunch" and choose "Set as StartUp Project".

And OpenSim will run from Visual Studio on 64-bit Windows...

SECS in technicolor

Just a model of the pipeline. Work in progress. Boxes that share same color are related.

Script Engine Component System (SECS)

What is a script engine?
Currently I see people define a script engine very simply as: The thing that runs the scripts.
This is a narrow definition that I never intended with DotNetEngine (see previous posts).

Instead I would say the script engine is the most important component of OpenSim. I am a humble person when it comes to my own doing and do not see DotNetEngine as anything more than a script executor. The greatness lies in the potential and what other people will make if we can offer them some good interfaces.

Instead of giving this complex explanation that required several of your neurons to connect I will use examples:

• Second Life LSL2 script support. Events and commands. Execution.
• OSSL script support. Other events and other commands. Execution.
• Additional command modules to for example give scripts SQL-access, Webservice acces, shared memory, etc.
• A plant module that grows grass/trees randomly around the region
• A bombterrain module that makes holes in the ground when bombs are detonated and fill them slowly back up over time.
• NPCs
• Bridges: Examples: "Contact us" module that allows sending of e-mail when someone drops a message in your region, IRC chat bridge, MSN chat bridge, etc.
• Complete game systems that allows a single system to control some or all aspect of the region/grid. Multiple modules can be loaded simultaneously to cover all aspects (one covers NPCs, others covers inventory stuff, swords, lazer guns, etc etc).

This was the original idea behind DotNetEngine, and the reason for the design of inheriting only what you need from the engine. I do not want OpenSim to be limited to in-world scripting only.

SECS
So how can we support all this?

Note! This is work in progress!

I imagine in the future we will want to download a good NPC module and install it independently of what script engine we are running. Lets face it, if each module had to be adapted to a certain script engine then it would put some heavy restrictions on us.

Instead I propose we redefine what a script engine actually is, and allow it to co-exist with other script engines. We may not want to run DotNetEngine and xengine simultaneously, but we may want to run a whole bunch of other "engines" (game control systems, AI-bots, etc) along with a script engine. And while these systems are running they should be able to benefit from the services the script engine already offers such as event hookups (into OpenSim) and the scheduler - IF they want.

To achieve this I have been looking at different models, all with their advantages and disadvantages. Though I have not landed at a perfect model yet I am closing in.

Basically it is a loader. It loads modules (.dll). The modules are registered in a central location with a name (string). Some of these modules will be of well-known base types (event hookups, schedulers, etc). Because they are well-known base types an alternate module should be able to utilize it even though it wasn't made for that distinct type.

When a script engine is loaded it will create instances of whatever it needs to do its job. This is pretty much exactly how the script engines work today. The only exception is that we (probably) need to register the instances globally.

Any other modules that are started are now free to either instanciate what modules they need. Example of what they may load: OSSL commands, SQL commands. They can also access main script engine scheduler if they want to associate a custom class with a prim for example.

Note! This is work in progress!

DotNetEngine and xengine

The realization
Before my hands went bad I started the process of introducing a new common way of implementing script engines. This came from the realization of two things. First of all that many had difficulties understanding how to implement their own engine variations based on current engine, and second that not all developers are willing to conform to one structure.

Developers are (often) skilled people and often have a clear understanding of what is right. Structure and rules on how to do things are key elements to write good code. And with the complexity of modular programming languages there is more than one way to get things done 'right'.

DotNetEngine
The idea behind DotNetEngine was simple. We implement an engine consisting of a set of modules and a couple of classes to bind it all (main ScriptEngine and in-world ScriptLoader). Any new script engine could then inherit what it liked and ignore what it didn't like.

DotNetEngine was made from the assumption that users are hostile (or stupid). They will run too many scripts, script may be harmful to system, etc. It works with a set of queues that ensures equal priority for all scripts and will drop loading/execution if system resources is exhausted (if you don't have a fast enough CPU). It allows things like regions themselves and physics to go in front of script execution.

I felt this approach was correct for a generic engine as it would guarantee a smooth region execution.

xengine
xengine is for the most part the effort of Melanie. It is targeted towards gaming in the sense that scripts essentially will have dominance over resources.

With the introduction of xengine the DotNetEngine modular inheritance model was bypassed and most of DotNetEngine simply copied into xengine and made modifications to. This means that any patch needs to be applied to two places, which of course meant double work and ended in it not being done. This was especially bad for LSL implementation where steady progress has been over a long period of time and suddenly there were demands that new functions should be implemented only in xengine.

DotNetEngine vs xengine
There has been an ongoing conflict between the developers on how to solve this problem. On one side we have the newly formed xengine mob which feels that DotNetEngine is inferior, and on the other side we have those who cheer for DotNetEngine and a revamp.

I've personally chosen not to take part in this debate as I agree with Einstein (with a modification) - Politics lasts for about a month, code lasts forever. I have however seen a lot of false assumptions been spoken both about DotNetEngine and about my self and my intentions, as well as a few true ones. But I believe this is totally irrelevant. We have the power to fix anything, and frankly most of the ones pointed out in DotNetEngine are not that big. However I feel some sides in this case may have be pushing too strongly on the political side and not doing too enough the code side to solve the problem - and that is a WOMBAT. (Waste Of Money, Brains And Time)

SECS
Now back to this common structure I mentioned at the start of this post.
--- to be continued in next post ---

Absent

Hello... Is this thing still on?

Ok, its a while between each post. The amount of time I have varies.

Lately I've been disabled by some inevitable hand-hurting as a result from effort in another project I worked on. I'm also in the startup phase with a new company, so time is still not something I have an abundance of - but who does.

After a nice scenic vacation in north of Norway my hands are 70%-80% back to normal and here I am. :)

OpenSim tackles high load

3Di have provided us with some new fancy features being added to OpenSim over the next few days.

Region splitting
Allows multiple physical computers to cooperate on maintaining one region. This is good for scenarios where you would want a large amount of people in the same region (think large events). OpenSim has already proven the potential of holding well over a hundred users, and this patch increases that number many times.

Dynamic load balancing
Monitors the load (often related to number of users) a region is exposed to and can realtime move the region to a computer with less load (while it's running!). More efficient use of computer resources means we save electricity. No more empty regions occupying an idle server. So this is a green patch! :)

Objectifying LSL

A few months ago we decided to make a distinction between SL's LSL2 and our own LSL implementation. This mainly because we were planning to add new features to the language. But also to make people understand that the compatibility may not be 100% - at least not for some time into the future. One thing is implementing all ll-commands. Another is to get them all right. So we decided to call our language OSSL (OpenSim Scripting Language.)

Yesterday OSSL took a new turn. In addition to LSL2's straight forward function calls and our own similar os-calls I want to provide a modular aproach.

So now OSSL has a new object "Prim" that can be used in the following ways:
Prim.Position.x += 10;
Prim.Rotation.x += 45;
Prim.Text = "This text floats on top of a prim!";

I'm planning to implement more functions. Especially looking forward to getting for example the particle-system into objectified managed code.

Later down the road people can use Visual Studio to write scripts for OpenSim. If they include some OSSL.dll they will get the benefits of managed code and writing scripts should be a breeze.

So, here is a sample script:

//c#

double x = 0;
double y = 0;

public
void default_event_state_entry()
{
llSetTimerEvent(0.1);
}

public void default_event_timer() {
x += 0.2;
y += 0.1;

Prim.Position.x = System.Math.Sin(x) * 10 + 100;
Prim.Position.y = System.Math.Cos(y) * 10 + 100;
}

New languages

DotNetEngine has received a couple of new languages: VB.Net and JScript.
Just add //vb or //js as the first characters of your script. Examples here: http://opensimulator.org/wiki/OpenSim:Scripting_Languages

LSL support has also been improved to support states and also a minor patch for functions inside . It seems like LSL is starting to run ok now.

New common framework

I have moved most of the code from OpenSim.Region.ScriptEngine.DotNetEngine to OpenSim.Region.ScriptEngine.Common.

Right now we have only one ScriptEngine, and its purpose is to emulate a Second Life script engine by executing scripts put into prims. But in the future I expect different kinds of script engines such as controllers that manipulate in-world game rules or objects from a .dll that has nothing to do with prim scripts.

Therefore I moved parts of DotNetEngine into Common so that Common is an execution framework.

DotNetEngine now does LSL->C# convert and compile. But does not do anything else. It simply hands the compiled .dll back to Common when its done. Then Common does the actual execution of script.

To quote http://opensimulator.org/wiki/OpenSim.Region.ScriptEngine.Common

What do I get for free?
1. Loading/unloading of scripts are queued and executed in sequence. Only one load/unload at the time.

2. What script belongs to what object is automatically taken care of. You just need to provide the scripts classes - nothing else.

3. OpenSim events are translated and given to you as LSL events. Your script functions are executed automatically, no need to hook up to events or anything.If you want to implement your own event handlers, feel free to do so. Look in "OpenSim.Region.ScriptEngine.Common.ScriptEngineBase.EventManager.cs" how its done. Your own events can run together with LSL events without any problem.

4. Whenever an event is executed, the execution is queued. Only one event per script is ever executed simultaneous. But multithreading is used to run multiple scripts in parallel.

5. Errors during script execution is automatically handled and relayed to users in-world.

6. You get all LSL commands in an easy to use object, no need to keep track of scene or objects to execute functions. Running an LSL command from your script engine is as easy as running it from inside any LSL-script.

7. Long LSL-commands that returns with an event is also handled.

ScriptEngine with many regions per server

I have added a whole lot of detailed config options in OpenSim.ini.example that can be used to control DotNetEngine. Many of these are specific to Common, some are specific to DotNetEngine.

You can now control a lot of detailed stuff that you usually wouldn't care about. The reason why these options are there now is because 1) Common will be sharing resources across different script engines (so we can support large amounts of script engines) and 2) who knows what you will be using OpenSim for.

One of the most important new features is the capability of Common to share maintenance, load/unload and execution threads accross instances (read: regions). Today a new instance of ScriptEngine is created for each region. And with 3-4 threads per ScriptEngine ... I guess this is a severe limit to large scale use of OpenSim.

Note! The thread sharing is currently disabled due to some bugs that needs fixing.