Tedd Maa

Blog moved

2010-06-25T22:07:00.002+02:00

Blog moved to http://blog.tedd.no/
OpenSim stuff: http://blog.tedd.no/index.php/category/opensim
OpenSim RSS Feed: http://blog.tedd.no/index.php/category/opensim/feed

OpenSim in Visual Studio on Win64

2008-12-05T19:00:00.002+01:00

Note! Article applies to developers using Visual Studio on 64-bit Windows.

Visual Studio is a great tool. Most developers who can use it should.

Currently OpenSim has some problems with 64-bit mode. This is because we have some native .dll's and .so's such as:

SQlite
ODE
OpenJpeg
+ more

SQLite you can do without by choosing MySQL or similar. But the rest is a bit more difficult.
See http://www.adamfrisby.com/blog/2008/08/running-opensim-under-a-64-bit-environment/ for more info.

To solve this problem we have a file named "OpenSim.32BitLaunch.exe" which is compiled with CPU target set to 32-bit. This in term loads OpenSim.exe as a standard module and executes it.

But this is only from command line. If you want all the glory of debugging that Visual Studio provides then this workaround will work:

Download the OpenSim source as usual, run prebuild as usual.
Open OpenSim.sln in Visual Studio.
Right click solution and choose Add->Existing Project...
Browse to "OpenSim\Tools\OpenSim.32BitLaunch" and add the project "OpenSim.32BitLaunch".
Right click project called "OpenSim.32BitLaunch" and choose "Set as StartUp Project".

And OpenSim will run from Visual Studio on 64-bit Windows...

SECS in technicolor

2008-10-01T15:59:00.003+02:00

Just a model of the pipeline. Work in progress. Boxes that share same color are related.

Script Engine Component System (SECS)

2008-09-27T15:10:00.004+02:00

What is a script engine?
Currently I see people define a script engine very simply as: The thing that runs the scripts.
This is a narrow definition that I never intended with DotNetEngine (see previous posts).

Instead I would say the script engine is the most important component of OpenSim. I am a humble person when it comes to my own doing and do not see DotNetEngine as anything more than a script executor. The greatness lies in the potential and what other people will make if we can offer them some good interfaces.

Instead of giving this complex explanation that required several of your neurons to connect I will use examples:

Second Life LSL2 script support. Events and commands. Execution.
OSSL script support. Other events and other commands. Execution.
Additional command modules to for example give scripts SQL-access, Webservice acces, shared memory, etc.
A plant module that grows grass/trees randomly around the region
A bombterrain module that makes holes in the ground when bombs are detonated and fill them slowly back up over time.
NPCs
Bridges: Examples: "Contact us" module that allows sending of e-mail when someone drops a message in your region, IRC chat bridge, MSN chat bridge, etc.
Complete game systems that allows a single system to control some or all aspect of the region/grid. Multiple modules can be loaded simultaneously to cover all aspects (one covers NPCs, others covers inventory stuff, swords, lazer guns, etc etc).

This was the original idea behind DotNetEngine, and the reason for the design of inheriting only what you need from the engine. I do not want OpenSim to be limited to in-world scripting only.

SECS
So how can we support all this?

Note! This is work in progress!

I imagine in the future we will want to download a good NPC module and install it independently of what script engine we are running. Lets face it, if each module had to be adapted to a certain script engine then it would put some heavy restrictions on us.

Instead I propose we redefine what a script engine actually is, and allow it to co-exist with other script engines. We may not want to run DotNetEngine and xengine simultaneously, but we may want to run a whole bunch of other "engines" (game control systems, AI-bots, etc) along with a script engine. And while these systems are running they should be able to benefit from the services the script engine already offers such as event hookups (into OpenSim) and the scheduler - IF they want.

To achieve this I have been looking at different models, all with their advantages and disadvantages. Though I have not landed at a perfect model yet I am closing in.

Basically it is a loader. It loads modules (.dll). The modules are registered in a central location with a name (string). Some of these modules will be of well-known base types (event hookups, schedulers, etc). Because they are well-known base types an alternate module should be able to utilize it even though it wasn't made for that distinct type.

When a script engine is loaded it will create instances of whatever it needs to do its job. This is pretty much exactly how the script engines work today. The only exception is that we (probably) need to register the instances globally.

Any other modules that are started are now free to either instanciate what modules they need. Example of what they may load: OSSL commands, SQL commands. They can also access main script engine scheduler if they want to associate a custom class with a prim for example.

Note! This is work in progress!

DotNetEngine and xengine

2008-09-27T14:32:00.006+02:00

The realization
Before my hands went bad I started the process of introducing a new common way of implementing script engines. This came from the realization of two things. First of all that many had difficulties understanding how to implement their own engine variations based on current engine, and second that not all developers are willing to conform to one structure.

Developers are (often) skilled people and often have a clear understanding of what is right. Structure and rules on how to do things are key elements to write good code. And with the complexity of modular programming languages there is more than one way to get things done 'right'.

DotNetEngine
The idea behind DotNetEngine was simple. We implement an engine consisting of a set of modules and a couple of classes to bind it all (main ScriptEngine and in-world ScriptLoader). Any new script engine could then inherit what it liked and ignore what it didn't like.

DotNetEngine was made from the assumption that users are hostile (or stupid). They will run too many scripts, script may be harmful to system, etc. It works with a set of queues that ensures equal priority for all scripts and will drop loading/execution if system resources is exhausted (if you don't have a fast enough CPU). It allows things like regions themselves and physics to go in front of script execution.

I felt this approach was correct for a generic engine as it would guarantee a smooth region execution.

xengine
xengine is for the most part the effort of Melanie. It is targeted towards gaming in the sense that scripts essentially will have dominance over resources.

With the introduction of xengine the DotNetEngine modular inheritance model was bypassed and most of DotNetEngine simply copied into xengine and made modifications to. This means that any patch needs to be applied to two places, which of course meant double work and ended in it not being done. This was especially bad for LSL implementation where steady progress has been over a long period of time and suddenly there were demands that new functions should be implemented only in xengine.

DotNetEngine vs xengine
There has been an ongoing conflict between the developers on how to solve this problem. On one side we have the newly formed xengine mob which feels that DotNetEngine is inferior, and on the other side we have those who cheer for DotNetEngine and a revamp.

I've personally chosen not to take part in this debate as I agree with Einstein (with a modification) - Politics lasts for about a month, code lasts forever. I have however seen a lot of false assumptions been spoken both about DotNetEngine and about my self and my intentions, as well as a few true ones. But I believe this is totally irrelevant. We have the power to fix anything, and frankly most of the ones pointed out in DotNetEngine are not that big. However I feel some sides in this case may have be pushing too strongly on the political side and not doing too enough the code side to solve the problem - and that is a WOMBAT. (Waste Of Money, Brains And Time)

SECS
Now back to this common structure I mentioned at the start of this post.
--- to be continued in next post ---

Absent

2008-09-27T14:17:00.004+02:00

Hello... Is this thing still on?

Ok, its a while between each post. The amount of time I have varies.

Lately I've been disabled by some inevitable hand-hurting as a result from effort in another project I worked on. I'm also in the startup phase with a new company, so time is still not something I have an abundance of - but who does.

After a nice scenic vacation in north of Norway my hands are 70%-80% back to normal and here I am. :)

OpenSim tackles high load

2008-03-04T21:02:00.004+01:00

3Di have provided us with some new fancy features being added to OpenSim over the next few days.

Region splitting
Allows multiple physical computers to cooperate on maintaining one region. This is good for scenarios where you would want a large amount of people in the same region (think large events). OpenSim has already proven the potential of holding well over a hundred users, and this patch increases that number many times.

Dynamic load balancing
Monitors the load (often related to number of users) a region is exposed to and can realtime move the region to a computer with less load (while it's running!). More efficient use of computer resources means we save electricity. No more empty regions occupying an idle server. So this is a green patch! :)

Objectifying LSL

2008-02-25T18:34:00.005+01:00

A few months ago we decided to make a distinction between SL's LSL2 and our own LSL implementation. This mainly because we were planning to add new features to the language. But also to make people understand that the compatibility may not be 100% - at least not for some time into the future. One thing is implementing all ll-commands. Another is to get them all right. So we decided to call our language OSSL (OpenSim Scripting Language.)

Yesterday OSSL took a new turn. In addition to LSL2's straight forward function calls and our own similar os-calls I want to provide a modular aproach.

So now OSSL has a new object "Prim" that can be used in the following ways:
Prim.Position.x += 10;
Prim.Rotation.x += 45;
Prim.Text = "This text floats on top of a prim!";

I'm planning to implement more functions. Especially looking forward to getting for example the particle-system into objectified managed code.

Later down the road people can use Visual Studio to write scripts for OpenSim. If they include some OSSL.dll they will get the benefits of managed code and writing scripts should be a breeze.

So, here is a sample script:

//c#

double x = 0;
double y = 0;

public
void default_event_state_entry()
{
llSetTimerEvent(0.1);
}

public void default_event_timer() {
x += 0.2;
y += 0.1;

Prim.Position.x = System.Math.Sin(x) * 10 + 100;
Prim.Position.y = System.Math.Cos(y) * 10 + 100;
}

New languages

2008-02-11T14:04:00.000+01:00

DotNetEngine has received a couple of new languages: VB.Net and JScript.
Just add //vb or //js as the first characters of your script. Examples here: http://opensimulator.org/wiki/OpenSim:Scripting_Languages

LSL support has also been improved to support states and also a minor patch for functions inside . It seems like LSL is starting to run ok now.

New common framework

2008-02-11T13:52:00.001+01:00

I have moved most of the code from OpenSim.Region.ScriptEngine.DotNetEngine to OpenSim.Region.ScriptEngine.Common.

Right now we have only one ScriptEngine, and its purpose is to emulate a Second Life script engine by executing scripts put into prims. But in the future I expect different kinds of script engines such as controllers that manipulate in-world game rules or objects from a .dll that has nothing to do with prim scripts.

Therefore I moved parts of DotNetEngine into Common so that Common is an execution framework.

DotNetEngine now does LSL->C# convert and compile. But does not do anything else. It simply hands the compiled .dll back to Common when its done. Then Common does the actual execution of script.

To quote http://opensimulator.org/wiki/OpenSim.Region.ScriptEngine.Common

What do I get for free?
1. Loading/unloading of scripts are queued and executed in sequence. Only one load/unload at the time.

2. What script belongs to what object is automatically taken care of. You just need to provide the scripts classes - nothing else.

3. OpenSim events are translated and given to you as LSL events. Your script functions are executed automatically, no need to hook up to events or anything.If you want to implement your own event handlers, feel free to do so. Look in "OpenSim.Region.ScriptEngine.Common.ScriptEngineBase.EventManager.cs" how its done. Your own events can run together with LSL events without any problem.

4. Whenever an event is executed, the execution is queued. Only one event per script is ever executed simultaneous. But multithreading is used to run multiple scripts in parallel.

5. Errors during script execution is automatically handled and relayed to users in-world.

6. You get all LSL commands in an easy to use object, no need to keep track of scene or objects to execute functions. Running an LSL command from your script engine is as easy as running it from inside any LSL-script.

7. Long LSL-commands that returns with an event is also handled.

ScriptEngine with many regions per server

2008-02-11T13:51:00.001+01:00

I have added a whole lot of detailed config options in OpenSim.ini.example that can be used to control DotNetEngine. Many of these are specific to Common, some are specific to DotNetEngine.

You can now control a lot of detailed stuff that you usually wouldn't care about. The reason why these options are there now is because 1) Common will be sharing resources across different script engines (so we can support large amounts of script engines) and 2) who knows what you will be using OpenSim for.

One of the most important new features is the capability of Common to share maintenance, load/unload and execution threads accross instances (read: regions). Today a new instance of ScriptEngine is created for each region. And with 3-4 threads per ScriptEngine ... I guess this is a severe limit to large scale use of OpenSim.

Note! The thread sharing is currently disabled due to some bugs that needs fixing.

ScriptServer

2008-02-11T13:46:00.000+01:00

ScriptServer is "almost done". Meaning it is capable of doing everything except communicating from a script to OpenSim. For this I need to abstract the layer between llFunction() and the actual implementation. Which already is in part done.
However ScriptServer has not been a priority lately.

For those who dont know the difference: ScriptEngine is a .dll that OpenSim uses to run scripts. ScriptServer is 1) a ScriptEngine .dll, but instead of executing scripts it remotely communicated with a 2) ScriptServer that uses the same ScriptEngine .dll's as OpenSim to execute scripts.
The advantage of this is that the script execution can happen on a remote machine (or same machine if you want) outside of OpenSim.exe.

Idling

2008-01-11T18:50:00.000+01:00

I guess its that time of year again when I write my blog post...

It's been a while... :) I've been busy with RL work. Sadly that work doesn't involve OpenSim just yet. But now I'm back and I thought I'd put some words here.

I've picket up where I left off on ScriptServer. Not so much to report yet. TCP server/client implementation is ready, binary protocol and bindings are not. Server executable is ready, plugin module is ready. So what remains is mainly binary protocol and binding implementations.

Depending on how much free time I have this shouldn't take too long.

Script engine implementation

2007-10-14T23:03:00.000+02:00

I just wrote this contribution to AWG: http://wiki.secondlife.com/wiki/Tedds_stand-alone_script_engine

It explains how and why on my opinions on the future script engine implementation.

Communication

2007-10-14T17:27:00.000+02:00

Things are a bit quiet right now. But there are things happening behind the scenes.

Some notes on communication between region and script engine.

Region will be the initiator. That is, the one that contacts script server. If an object containing a script moves from one region to another then the next region will contact the script server.

The communication protocol will be a custom binary protocol. As far as I know there are no RPC protocol designed for high volume async function calls. This will allow a continous stream of function calls in both directions. Any reply on commands will be done by attaching an reply ID to the function call package, a reply package refering this ID containing reply data will be sent back.

Scripts will be blocked while waiting for reply, but not while waiting for their command to be sent.

Currently TCP client/server has been created, but the binary protocol is still in design stage.

A peek on new architecture

2007-10-01T19:11:00.000+02:00

This is where my planning is at now.
It was very complex, and after redrawing it a few times I neded up at this.
Still not 100%, but getting there...

Major rewrite number 2

2007-09-27T18:21:00.000+02:00

I have been doing some thinking on the ScriptEngine. Actually a whole lot of thinking, and planning. Lots of alien squid-like drawings in my notebook.

Trying to run through the process: Idea, required elements, architecture and finally simplification.

The idea is simply: Moving ScriptEngine to a separate daemon, independent from the region.
This has several big advantages and of course some drawbacks.

Script does not need to be moved between regions when object it is inside does.
Scripts can run on separate servers and even clusters of servers, offloading the main region server.
Script source (and binary) can be secured by running it from a trusted server. Even makes it possible with script rental. You can run your own ScriptEngine.
We can add the option to move script between regions as usual to avoid network lag on time-critical scripts.
It is easier to write alternative ScriptEngines, developers are not restricted to .Net.
ScriptEngines can be more complex and allow bigger and more complex scripts than the region would by default.
Regions can be more secure as they do not have to execute scripts locally.

Disadvantage:

Each region may have to speak to many different scriptengines.
Network latency will affect script performance.
Script reliability is much lower.
Various problems related to authentication.

Of course all the work done with implementing LSL commands so far will survive. That is independent of this rewrite.

I'll be back with more as it unfolds.

Perfect code

2007-09-27T18:13:00.000+02:00

The development process of a good program may often seem strange for an outsider. One can expect the development process to consist of planning, development and testing. A simple linear way to get from design to final product. In most commercial scenarios this is also how its done. Forcing an application from idea through design, programming and testing to the final product. And this is right, because in almost every case a program is design to do a specific task. There are rarely and demands on how the code looks or how secure it is. The application will be serving 1, 2 or 50 people for 2-3 years with little modifications once its has been developed.

However when a programmer who loves to code is sitting at home developing something the process is very different. Programs are rarely finished, they are often only 70-80% done before scrapped and a new one is started. Debugging has been done during development, and the program is likely to contain lots of bugs. Because the program was not written for the purpose of being used. It was a practice... A practice in architecture, algorithms, API's and objects. The next time the a program is written the past experience is put into that program, and its slightly better. It's all about understanding the architectures and algorithms. Therefore, in search for the "perfect code", you find developers rewriting their applications - time after time. Sometimes it's major restructuring, sometimes major rewrites. Learning how to plan for change, so that you don't need to plan.

Do I need to say more? nah.

Pure C# support

2007-09-14T10:06:00.001+02:00

ScriptEngine supports both LSL and C# within the same script. Mainly this means that you can write LSL, but when you want you can use C# code. For example instead of using llToLower(mystring) you can use mystring.ToLower(), instead of llStringLength(mystring) you can use mystring.Length().

But to make sure that we support pure C# code, I've added support for forcing the compiler to not use LSL preprocessor. This means skipping directly to compile without going through LSL to C# converter.

Any script that starts with the 4 letters "//C#" or //c#" will be compiled as C# directly. This ensures that we have 100% C# support.

There are a few rules you need to follow. Mainly:

Any LSL event needs to be renamed to <STATE>_event_<EVENTNAME>.
For example "start_entry()" in default state will be "public void default_event_state_entry()"
LSL events needs to be public, of course.
Script needs to start with "//C#" without the quotes and without space in front of it. These 4 characters needs to be the first thing inside the script!
You still have to use ll-commands for stuff that native C# doesn't support. But don't worry - the ll-commands are just the name, you are actually writing in C#. ;)

If you put a LSL script into an object in OpenSim, the C# code of that script will be debug-logged to the "bin\ScriptEngine\"-folder. So if you have any trouble, just write up a LSL code, put it into an object and you have a working C# script that does the same thing.

Untested example:

//C#

namespace SecondLife
{

public class Script : OpenSim.Region.ScriptEngine.DotNetEngine.Compiler.LSL.LSL_BaseClass
{

public Script() { }

      // Your script goes here!
      int touch_count = 0;
      int tick_count = 0;
      int objnum = 1;

      public void default_event_state_entry()
      {
         llSay(0, "Hello, Avatar! Starting timer...");
         llSetTimerEvent(10);
      }

      public void default_event_touch_start(int total_number)
      {
         touch_count++;
         llSay(0, "Object was touched. Touch count: " + touch_count);
      }

      public void default_event_timer()
      {
         tick_count++;
         llSay(0, "Timer count: " + tick_count);
      }
   }
}

Starting my own company

2007-09-09T10:32:00.000+02:00

Hi

I'm currently in a period of less activity on the development front. Very sorry about that.
At the end of last month I quit my job and started my own company as an independent consultant, and with this comes a lot of work setting up, getting customers, etc...

The good news is that once this is through I will be back for full. The plan is that my company will support this project through my development effort, so there will be time set of specially for OpenSim. But the time I can put into it will of course vary a bit depending on what projects I'm on.

But I'm looking very much forward to starting back up on OpenSim. The plan is tomorrow (monday) or tuseday, depending on finalization of current project. But my mind has not been completely absent, I've been thinking about the ScriptEngine allthough I haven't had time to do any coding.

This is what I want to fix next:

Look into how to move ScriptEngine from Region to Sim-level with MW, changing object and script identifiers to LLUUID.
This way a script can keep running even though it crosses local regions - a great benefit to both the script itself and the servers.
Document how so anyone who wants to take on the mission of creating llHttpRequest can do so. (this is one of those long commands that needs cleaning up if a script is removed)
Serialize/Deserialize code in AppDomains so we can move running scripts (that are inbetween executions) and keep their state when stopping/starting server.
Make scripts work in Grid-mode.

Also I have been playing with the idea of restructuring ScriptEngine so that it can easily be used in other projects too. But we'll have to wait and see...

WIKI updated with info on scripting

2007-08-30T09:48:00.000+02:00

I've updated the OpenSim Wiki with:

General info about scripting in OpenSim
llFunction implementation status
And some information about the DotNetScriptEngine internals

Small stresstest

2007-08-26T00:09:00.000+02:00

114 scripts loaded. 1 script per AppDomain. Most scripts are alone inside a prim, except the big box over there that has something like 10 scripts in it (it is bigger so I could fit more scripts into it).

My computer is a 2,8GHz Prescott with 2GB RAM and Windows Vista Ultimate.
60 of the scripts run the timer event every 1 second. 50 run it every 5 seconds.

Basically this tells us that ScriptEngine uses a bit too much memory. This is expected. CPU usage while running 114 events every x seconds is almost nothing.

Increase from 38MB to 76MB RAM when creating objects and scripts, so ~10MB for 25 scripts+objects.

The scripts don't do much work right now, but if they did the speed would still be high as they are compiled to CPU native code.

Results after running for 9 hours

OpenSim did not crash or eat any more CPU or memory over time. However the Second Life viewer did crash, so I had to log in again... Funny ;)

So where are we now?

2007-08-25T22:55:00.000+02:00

[Updated Monday 13:45 GMT+1]
[Updated Wednesday 21:45 GMT+1]
[Updated Saturday 22:45 GMT+1]

As always its important with a summary for those who don't want to read techie details.

Here is current status of LSL script support:
+ We have LSL script support.
+ We can run hundreds of scripts simlutaneously, probably not thousands (needs testing). We have confirmed 50 simultaneous scripts without any sign of trouble.
+ We can utilize multicore CPU's for running scripts.
+ We can to some degree unload scripts to free up memory. (if we use 1 AppDomain per script)
+ Events from server are put into a queue, so server threads are released imediately.
+ Some llFunctions are implemented.
+ ScriptEngine is more or less ready to bind individual scripts to individual prims.
+ We have implemented touch_start event when someone touches a prim.
+ We have implemented llSetTimerEvent.
+ Script compile errors are displayed in-world.
+ We can put scripts into prims.

- Can not run too many scripts that contain infinite loops or sleeps to long.
- No memory/CPU sharing/limitation/throttling on scripts.
- Almost no security implemented on scripts.
- Can not move script between regions.
- Missing 300+ llFunctions.
- Missing most events.

There is room for making scripts run 2-5 times faster, if not more. And same in memory usage. This is because of the development process.
"Get it working, remove bugs, make it fast."

There is much data related to each script. I suspect if we load too many scripts the working set (WSClock) will be too high. Meaning there is too much data to process, so the CPU and memory will be more occupied with looking at what to do than actually doing it.

Introduction of a new class

2007-08-25T22:45:00.000+02:00

Some LL-commands takes longer to return. These often return with a event. For example llSetTimerEvent(10) will return a timer() event every 10 seconds. llHttpRequest() will return when it receives answer from HTTP-server.

To handle these types of requests a new class has been added as a subclass of ScriptEngine. The class is named LSLLongCmdHandler.cs. This class will contain handlers for adding/removing requests. Adding when script executes a command, removing when command is not complete yet but script is being moved to another region or otherwise removed.

The class has a dedicated thread that will be working the different services, for example async HTTP requests and queueing of timer events at the correct intervals.
LSL_BuiltIn_Commands.cs accesses this class through ScriptEngine.

Example from LSL_BuiltIn_Commands.cs:


public void llSetTimerEvent(double sec)
{
  // Setting timer repeat
  m_ScriptEngine.myLSLLongCmdHandler.SetTimerEvent(m_localID, m_itemID, sec);
}

Speed

2007-08-19T15:59:00.000+02:00

I've added support for multithreading execution of script events. Only one object will be processed at the time. This will benefit from MultiCore CPU's. Though it makes each call more expensive because we have to keep track of mutexed objects.

Also added caching of event binding (function call) so we only use Reflection on first execution of a given event within a given state.

Changed sleep time of inactive threads from 200ms to 50ms, so they are more responsive.

AppDomains

2007-08-19T15:46:00.000+02:00

.Net does not allow unloading of assemblies. If you want to unload an assembly you have to unload the whole AppDomain.

I've created AppDomainManager that handles creating AppDomain, finding a free AppDomain, loading scripts into it, and unloading AppDomains with no active scripts in them.

Its purpose is to create AppDomains and put X number of scripts into each of them. Later when we get microthreading support we will add a feature to move scripts between AppDomains, so if one or more AppDomains has only a few active scripts we can move these to a fresh AppDomain and unload the old one.

With the introduction of AppDomain there came a few changes to the source. Because of how communication between AppDomains (mainly between scripts AppDomain and the main AppDomain) works there has been made some changes.

First of all a new assembly called OpenSim.Region.ScriptEngine.Common.dll has been introduced. It contains interface for scripts and some common code for all AppDomains to execute events inside scripts.

Secondly the LSL BuiltIns (llFunctions) has been moved to a separate file (LSL_BuiltIn_Commands.cs) and are no longer part of the LSL_BaseClass.cs.

Also OpenSim.Region.ScriptEngine.Common.LSL_Types that contains custom Vector and Rotation objects is now used throughout DotNetEngine/Compiler.

Each AppDomain has only 2 references, OpenSim.Region.ScriptEngine.Common.dll and OpenSim.Region.ScriptEngine.DotNetEngine.dll.

llSay implemented

2007-08-17T23:42:00.000+02:00

Picture of llSay() implemented.

llFunction interface/empty prototypes

2007-08-14T21:12:00.000+02:00

Charles Krinkle has helped in adding interface and necessary empty prototype implementations of base class for the compiled script.

Almost all functions has been implemented (they contain no code yet though). This helps me test the compiler on all kinds of scripts without getting errors.

Thanks dude! :)

New status after Monday (and a milestone)

2007-08-14T09:00:00.000+02:00

ScriptEngine has now been implemented!

We have a default script that is placed in all objects. We have a "touch_start" event. And we have llSay() with output to server console.

Once inventory is working we can bind user made scripts to user made objects. So now we can start implementing all 350+ builtin ll-functions.

We will be able to test the functions in-world as they are being implemented. Just modify the script "Default.lsl" under "bin\ScriptEngines\" folder to use the new command you are implementing.

There may still be problems compiling some scripts. Please let me know about these problems so I can fix them!
Next step for me is to implement error handling around ScriptEngine.

Status of LSL compiler

2007-08-12T22:19:00.000+02:00

Status update

LSL->C# Translator is close to complete. Needs bigger/more scripts to bughunt.
Compiler is capable of compiling C# to .Net (invokes LSL->C# trans if needed).
Script Manager is capable of loading and initializing script .Net Assembly.

Missing:
LSL BuiltIn command implementation.
Event implementation.

How to contribute?

2007-08-12T20:20:00.000+02:00

OpenSim.Region.ScriptEngine.DotNetEngine.Compiler.LSL.LSL_BaseClass.cs contains a s*load of empty functions. These are all the LSL-functions. And all needs to be implemented. Implement most important/most used first.

There is a "World" object available (scene).

Example of implementation:


public void llSay(Int channelID, string text)
{
   World.Something.Somewhere.SendMessageToEveryoneAround(channelID, text);
}

LSL to C#

2007-08-12T20:15:00.001+02:00

To compile LSL code I figured the best way would be to first translate it into C#.
Since LL will be supporting both LSL and C# we can use the same translator on a web page or similar to help people who know LSL to understand C#.
Also it is much simpler for other developers on OpenSim to contribute to the code when its C# instead of MSIL.

The translator uses some regex and a couple of byte-by-byte passes over the code. Written in less than a day, no big magic involved.

In-memory compile to assembly (that will be cached for later reuse).
Assembly is dynamically loaded in same way as LSO.

The script inherits from a LSL_BaseClass.cs that contains all the builtin-functions like llSay(), llWhisper, etc.

Changing tactics

2007-08-12T20:07:00.000+02:00

After a meeting with Babbage Linden in Zero Lindens office hours on thursday I realized that with the new Mono/.Net compiler they are implementing we will have to make a few adjustments.

It was earlier assume that compilation would be done at client of both LSL and LSO, therefore I focused on making a LSO (LSL virtual machine) ByteCode to .Net CIL (Assembly language) translator.

But because of some security concerns in the .Net ByteCode LL has chosen to do compilation on server. LSL or C# code is sent to server where it is compiled and put into an object. The compiled code is never sent from the viewer.

Therefore I am now writing a LSL compiler that will run on server.

While waiting for Linden to update their client (probably 0,5-1 years) we will use the text-copy of LSL script sent by viewer and compile that. Meaning we are not using the LSO at all.

So basically we are scrapping the LSO compiler, and going for a LSL compiler. There are many clear advantages to this, so I'm quite happy about it.

Small milestone reach today

2007-08-05T16:26:00.000+02:00

Application is now able to:

Create Script Engine (SE)
SE hooks up to a temporary debug event in Test App
Test App asks SE to load LSO-file
SE compiles LSO-file into .Net Assembly
SE loads .Net Assembly
SE Initializes the .Net Assembly and hands over a "LSL BultIn Commands" object (this object is the scripts command-object)
Script Manager adds Script to an Object[].Script[] structure
By pressing a button in Test App an event is sent to SE
SE receives event and queues it to correct ObjectID + ScriptID
Separate thread dequeues event and sends it to Script Manager
Script Manager executes function
llSay() is displayed to debug console

Status on initial SE implementation

2007-08-05T09:53:00.000+02:00

With the model described in the two prior postings we will have simple scripts support up and running fairly quickly.

Other developers will be able to extend script support while I continue working on Compiler and ScriptEngine. Both incoming events and outgoing LSL builtin commands needs extending.

We will lack script yield inserting. This is where the preparation of .dll for microthreading will happen.

We will have a simplified version of Event Queue Manager that will use a custom threadpool to execute events and brutally kill any long-running events.

We will havea simplified version of AppDomain handling that basically just loads all scripts into one AppDomain with no code security.

We will have a basic LSL Compiler that can compile the simplest LSL scripts, but that others can extend command support for.

In short, the scripts should be able to llSay() during rez. But can easily be extended by anyone to support the commands we need. The basic structure is in place, and we can build upon that bit by bit.

ScriptEngine - Events

2007-08-05T08:25:00.000+02:00

The builtin commands for SE is simple enough. But incoming events could need some more planning.

SE will have a set of events that is fired based on stuff that happens in OpenSim (EventHandler). This EventHandler will translate the event into a LSL-compatible event, including picking up the required parameters.

The EventHandler needs to know what object, and possibly what script, the event is for. Then it will queue this event for the correct script(s) (multiple scripts may subscribe to the same event in an object).

A Event Queue Manager will process the event queue under a separate thread and execute the correct script functions with the correct parameters. By using a separate thread OpenSims event thread is free to return immediately.

What scripts subscribe to what events is determined during script load. Either by dynamically identifying function names, or by having the script hook itself up during load. The functions will be stored by reference to avoid using expensive Reflection-calls for every script event.

One possiblity for extra speed is to dynamically (using CIL) create a call object in same AppDomain as script is located that does some of Event Queue Managers work.

ScriptEngine

2007-08-04T20:43:00.000+02:00

The time has come to start implementing script support into OpenSim.

ScriptEngine (SE)
OpenSim will have an interface for SE. It will load SE dynamically using reflection. SE will be in separate .dll Assembly.
During load SE will hook itself up to the necessary events in OpenSim.
This includes: Object rez (look for active scripts), object derez and all events relating to objects (touch, sit, etc).

SE might also during startup need to iterate through existing objects to see what scripts it needs to start. This depends on at what point SE is loaded.

All access to OpenSim from scripts will go through SE.

Compiler
When SE starts a script it will determine script type. If script type is LSO (LSL ByteCode) the script will be compiled into .Net Assembly .dll before load. Compiler is a separate .dll Assembly, but hardcoded into SE.

Loading scripts
* If script is LSO, compile to .Net Assembly
* Modify .Net Assembly to support microthreading (will be done later in project)
* Find available AppDomain for the script (currently only default appdomain)
* Load script into AppDomain
* Create a private instance of BuiltIn commands for the script
* Create object of script handing it the private BuiltIn command instance

Some BuiltIn functions will require information about who is executing them, for example for object permissions or feedback events. Because .Net Assembly can not be trusted (it is uploaded by users) ScriptEngine will create a private instance of BuiltIn commands. This instance will contain and pass on the necessary ID's to OpenSim - as well as work as a translator between LSL-functions and OpenSim API.

Operation
A script instance is made up of ScriptID and ObjectID. This should always result in an unique combination for any script.

ScriptEngine will receive events from OpenSim and forward the events to the required scripts. During this forward it will also enforce microthreading.

Current Status
Class OpenSim.ScriptEngine.DotNetEngine.ScriptEngine has basic framework and is able to load and execute scripts. Only missing event hookups and actual implementation into OpenSim.

Current status

2007-07-27T19:29:00.000+02:00

Static and global variable support has been added. All push/pop functions added.

Simple scripts that use variables, math and internal functions (like llSay) are now supported.
Stack and all internal data are ready for serializing, so running script can be moved between servers - but no fibre/user space multitasking/microthreading has been implemented yet - so move has to happen between events.

Missing support for local variables, custom functions, state, if-statements, jumps, receiving/pulling status on events, etc.

So.. We can start implementation into server! :)
In first version we'll run events using a thread. Maybe adding a timeout to the thread so that it will not run too long per script.

Current OPCODE implementation status

2007-07-26T22:40:00.000+02:00

This is the current OPCODE implementation status:


OPCODE                  IL_Processor         Implemented

NOOP                    *                    *
POP                     *                    *
POPS                    *                    *
POPL                    *                    *
POPV                    *                    *
POPQ                    *                    *
POPARG                  *                    *
POPIP
POPBP
POPSP               
POPSLR
DUP
DUPS
DUPL
DUPV
DUPQ
STORE                   *                    *
STORES                  *                    *
STOREL                  *                    *
STOREV                  *                    *
STOREQ                  *                    *
STOREG                  *                    *
STOREGS                 *                    *
STOREGL                 *                    *
STOREGV                 *                    *
STOREGQ                 *                    *
LOADP                   *                    *
LOADSP                  *                    *
LOADLP                  *                    *
LOADVP                  *                    *
LOADQP                  *                    *
LOADGP                  *                    *
LOADGSP                 *                    *
LOADGLP                 *                    *
LOADGVP                 *                    *
LOADGQP                 *                    *
PUSH                    *                    *
PUSHS                   *                    *
PUSHL                   *                    *
PUSHV                   *                    *
PUSHQ                   *                    *
PUSHG                   *                    *
PUSHGS                  *                    *
PUSHGL                  *                    *
PUSHGV                  *                    *
PUSHGQ                  *                    *
PUSHIP
PUSHBP
PUSHSP                  *                    *
PUSHARGB
PUSHARGI                *                    *
PUSHARGF                *                    *
PUSHARGS                *                    *
PUSHARGV
PUSHARGQ
PUSHE                   *                    *
PUSHEV
PUSHEQ
PUSHARGE                *
ADD                     *                    *
SUB                     *                    *
MUL                     *                    *
DIV                     *                    *
MOD                     *                    *
EQ                      *                    *
NEQ                     *                    *
LEQ                     *                    *
GEQ                     *                    *
LESS                    *                    *
GREATER                 *                    *
BITAND                  *                    *
BITOR                   *                    *
BITXOR                  *                    *
BOOLAND                 *                    *
BOOLOR                  *                    *
NEG                     *                    *
BITNOT                  *
BOOLNOT                 *
JUMP
JUMPIF
JUMPNIF
STATE
CALL
RETURN                  *                    *
CAST
STACKTOS
STACKTOL
PRINT
CALLLIB
CALLLIB_TWO_BYTE        *                    *
SHL
SHR

Book: CLR via C#, Second Edition

2007-07-26T09:30:00.000+02:00

This morning I received the book I ordered from Amazon.com. 1 week before estimated arrival.

The book is named CLR via C# (Second Edition) and is supposedly the bible of understanding CLR.

So 648 pages from now I should be able to utilize my new skills in the LSO project... :)

Looking ahead, fibres

2007-07-21T21:26:00.001+02:00

Running thousands of scripts simultaneously on a grid requires some special programming. We can not just issue one thread per script and let it run. The resources required by each OS thread,both in terms of memory and thread switching is too high.

Instead we want to execute a few instructions of a script, then move to the next script. All status about execution location, memory, etc. must be kept within each script. So when the execution thread is moving between scripts it carries no payload from any of the scripts.

An important thing about computer technology is learning about the past. Because old technology is recycled all the time. In this case one could look at the "multitasking" done in Windows 3.11 to pick up a few ideas. Or Windows NT's "fibre" support, a lightweight user space implementation of threads. And that is what it all boils down to, user space threading.

The clue here is to insert code into the script that will save it's position, mark next spot with a "label" so it can jump to that position later, and exit execution. (details will follow once I get this far in the project)

There are two points where I can insert this data. Either during convert, or before load. Before load would be preferred as we then could load any .Net Assembly - great for future .Net scripting support. But the easiest way is during convert. And since this is Version 1 I will follow the easiest road.

Frames, heap and code pointers

2007-07-21T21:16:00.000+02:00

LSL ByteCode uses a lot of pointers, both to memory (static frames, local/global, heap) as well as JUMP pointers.

Some of these pointers exist during load, for example argument types, static data, jump positions, etc. While others are allocated and created dynamically during run-time. What they all have in common is that they point to a position in the 16k LSO ByteCode file.

For memory pointers I'm planning to use a dictionary (or possibly an array), where key will be position and data the actual datastructure.

For JUMP positions I will need to scan the code first for JUMP's, then create MSIL labels in the code at these positions. This might require a two-pass run on the ByteCode.

Stack issues

2007-07-21T21:07:00.000+02:00

In my first attempt I used the .Net stack and did a direct translation from the LSO ByteCode. I found that there are a couple of problems related to that.

First of all is the obvious problem that I stated earlier, I don't know what datatype is in the stack - but many (most) .Net commands requires me to know that. Wherever LSO can provide datatype for me it is ok, but in many cases LSO is not providing me with that datatype. For example during casting.

Next is the problem that .Net wants the stack served like "classlocation, param1, param2, call to command in classlocation". This means that I need to predict commands before PUSH'ing anything to the stack. Way to complex. I could of course juggle the stack, but not so cool.

So I've made a few changes.

First of all I'm now inheriting the compiled .Net assembly from a baseclass. In this baseclass I've added my own Stack(object), and created my own (generic) PUSH/POP/POPToStack commands. (PUSH to my stack, POP from my stack and POP from my stack to .Net's stack)

Secondly I'm creating most OPCODES as commands in C# (the base class) that is called from the converted LSO ByteCode. This way I have full control of both stack and OPCODE behaviour.

The bonus is that my stack now is in a single object that can be serialized.

LSL scripting support - Progress report

2007-07-14T10:07:00.000+02:00

Ok, time for status update again...

Status

Code is currently able to convert from LSL ByteCode to .Net IL, compile it and save it to disk.
I've moved the code from a quick and dirty hack (that compiled the first script) to a bit more organized. It's still a mess, but that can be fixed later. The goal is simply "Version 1: Functional.".

So basically I'm able to convert simple LSL ByteCode-files into .Net .dll. If you load the script (.dll) into Visual Studio you will see the events (from server to script). If you provide the script (.dll) with an object with the righ inheritance the script (.dll) will use that objects LL-functions. Currently only llSay(); is implemented (mapped to Console.WriteLine). This is how the script communicated with the server.

I've added support for a few more OPCODES, so we support basic math and most PUSH/POP functions. Because LSL ByteCode based itself on pure memory addressing and .Net CLR/IL is object oriented a few of the functions will be tricky to implement. For example LSL wants to push a string followed by xx 0x000's to memory, then perform an operation and remove two bytes.

Server to script

For each event in the script I create one function that the server can execute. These functions needs to be late bound during load of .dll. I'm dynamically creating functions for each state+event combination. Because of the naming scheme, and the fact that a script seldonly listen on all events, I've not used any interface for this. The available events can be fetched through a GetEvents() functions that is hardcoded into the assembly when script is converted.
Also the required parameters for each function is created dynamically from the information in the LSO-file.

The naming scheme will be different once I'm done. Maybe something like a switch() inside each event to determine what code chunk should be executed based on what state number we are in.


UInt32 LSLObj.State; // Contains current state number
public void 0_event_state_entry() {} // state_entry event in state number 0
public void 1_event_state_entry() {} // state_entry event in state number 1
public void 0_event_touch_start() {}  // touch_start event instate number 0

Script to server

I've created an interface for all LSL builtin commands, and another class that inherits the interface. All the functions are there, but they are empty and don't have the correct arguments. Anyone can help to add commands by changing this class. This is probably the single remaining area that requires most work. (not sure though)

Since most PUSH/POP and math commands are supported already, we just need to map the builtin function calls to the above mentioned class. This is a simple Switch()-case hardcoding issue.

TODO:
From the top of my head, not in any particular order.

Support custom functions (currently only LL server events gets compiled)

Compiling them is easy, but I need to be able to bind them to Call OPCODE somehow. (IL issue)

Figure out how to reference the LSL builtin commands C# class from IL (IL issue)

Then bind all ll* functions

Add support for remaining OPCODES (IL issue)

This can be a bit tricky since LSL uses memory addressing in stack while .Net IL is object oriented. One possible solution is to implement same memory scheme as LL is using. Either way it's not a big problem, just a bit of work.

Figure out what OPCODES needs special (custom) commands in order to support interregional transfers and user space multitasking - instead of running CLR/IL OPCODE we run a C# function that does the required work.
Figure out how one can cast an element in CLR/IL stack without knowing its type, or - keeping track of all types in stack.

LSL is a bit like "push string, push long, convert to string, concat, push long(channel id), llSay()". Its asking me to cast whatever is on the stack, but CLR/IL requires me to know what type I'm casting from.

Figure out how to clear CLR/IL stack. CLR/IL requires stack to be empty on return. LSL sometimes leaves items in the stack. For now a Try:Catch is handling the issue.

The full source is here: http://www.konge.net/OpenSim/OpenSim.Region.Scripting_r1.zip.
Please note that this source belongs to OpenSim project (http://opensecondlife.org/wiki/OpenMetaverse) and is licensed thereafter.

Download, run project, choose a LSO-file (for example CloseToDefault.LSO), then process it.

Making some progress

2007-07-12T22:48:00.001+02:00

Today I found a "final bug" in my LSL ByteCode parser, and was able to get read seemingly clean ByteCode. I also made a quick hack of merging the CLR RunTime code generator with the LSL ByteCode parser, and compiled my first small function.
It works! :)

Also writing the compiled script to a .dll-file. Necessary as the only way I know of to debug CLR is through the loveable PEVerifier.EXE.

Of course most work remains. States are not supported, dynamic function names not supported, only one OPCODE (PUSHS - push string to stack) and only one BUILTIN command (llSay mapped to Debug.WriteLine) is supported.

Not to mention user space multitasking and moving a running script between regions.

Notice the last two lines of this log...


Debug: Opening filename: LSO\CloseToDefault.lso
Debug: Reading HEADER BLOCK at: 0
Debug: TM - Top of memory (size): 16384
Debug: IP - Instruction Pointer (0=not running): 0
Debug: VN - Version number: 512
Debug: BP - Local Frame Pointer: 16383
Debug: SP - Stack Pointer: 16383
Debug: HR - Heap Register: 233
Debug: HP - Heap Pointer: 240
Debug: CS - Current State: 0
Debug: NS - Next State: 0
Debug: CE - Current Events: 0
Debug: IE - In Event: 0
Debug: ER - Event Register: 0
Debug: FR - Fault Register: 0
Debug: SLR - Sleep Register: 0
Debug: GVR - Global Variable Register: 100
Debug: GFR - Global Function Register: 100
Debug: PR - Parameter Register: 0
Debug: ESR - Energy Supply Register: 0
Debug: SR - State Register: 100
Debug: NCE - 64-bit Current Events: 1
Debug: NIE - 64-bit In Events: 0
Debug: NER - 64-bit Event Register: 5
Debug: Read position when exiting HEADER BLOCK: 100
Debug: Reading STATIC BLOCK at: 100
Debug: Number of Static Blocks read: 0
Debug: No FUNCTION BLOCK found
Debug: Reading STATE BLOCK at: 100
Debug: Reading STATE POINTER BLOCK 1 at: 104
Debug: Pointer: 116
Debug: Total potential EventMask bits: 64
Debug: Reading STATE BLOCK 1 at: 116
Debug: State block Start Pos: 116
Debug: State block Header Size: 5
Debug: State block Header End Pos: 121
Debug: Reading STATE BLOCK 1 HANDLER matching EVENT MASK 0 (state_entry) at: 121
Debug: Reading STATE BLOCK 1 HANDLER EVENT MASK 0 (state_entry) Code Chunk Pointer: 137
Debug: Reading STATE BLOCK 1 HANDLER EVENT MASK 0 (state_entry) Call Frame Size: 0
Debug: Reading STATE BLOCK 1 HANDLER matching EVENT MASK 2 (touch_start) at: 129
Debug: Reading STATE BLOCK 1 HANDLER EVENT MASK 2 (touch_start) Code Chunk Pointer: 181
Debug: Reading STATE BLOCK 1 HANDLER EVENT MASK 2 (touch_start) Call Frame Size: 4
Debug: Reading Event Code Chunk state 0, event state_entry
Debug: CLR:event_state_entry:MethodBuilder methodBuilder = typeBuilder.DefineMethod...
Debug: CLR:event_state_entry:typeBuilder.DefineMethodOverride(methodBuilder...
Debug: CLR:event_state_entry:ILGenerator il = methodBuilder.GetILGenerator();
Debug: Reading Function Code Chunk at: 137
Debug: CodeChunk Header Size: 5
Debug: Function comment:
Debug: Return type: 99
Debug: Reading Code Chunk Argument 1
Debug: Code Chunk Argument 1 return type: 94
Debug: OPCODE: NOOP
Debug: OPCODE: NOOP
Debug: OPCODE: NOOP
Debug: OPCODE: PUSHARGS
Debug: Param1: Hello, Tedd!
Debug: OPCODE: PUSHARGE
Debug: Param1: 0
Debug: OPCODE: PUSHSP
Debug: OPCODE: PUSHARGI
Debug: Param1: 8
Debug: OPCODE: ADD
Debug: Param1: 17
Debug: OPCODE: POPBP
Debug: OPCODE: CALLLIB_TWO_BYTE
Debug: Param1: 23
Debug: OPCODE: RETURN
Debug: Last OPCODE was return command. Code chunk execution complete.
Debug: CLR:event_state_entry:il.BeginCatchBlock(typeof(Exception));
Debug: CLR:event_state_entry:il.Emit(OpCodes.Ldstr...
Debug: CLR:event_state_entry:il.Emit(OpCodes.Call...
Debug: CLR:event_state_entry:il.Emit(OpCodes.Callvirt...
Debug: CLR:event_state_entry:il.Emit(OpCodes.Call...
Debug: Reading Event Code Chunk state 0, event touch_start
Debug: CLR:event_touch_start:MethodBuilder methodBuilder = typeBuilder.DefineMethod...
Debug: CLR:event_touch_start:typeBuilder.DefineMethodOverride(methodBuilder...
Debug: CLR:event_touch_start:ILGenerator il = methodBuilder.GetILGenerator();
Debug: Reading Function Code Chunk at: 181
Debug: CodeChunk Header Size: 5
Debug: Function comment:
Debug: Return type: 99
Debug: Reading Code Chunk Argument 1
Debug: Code Chunk Argument 1 return type: 94
Debug: OPCODE: NOOP
Debug: OPCODE: NOOP
Debug: OPCODE: NOOP
Debug: OPCODE: PUSHARGS
Debug: Param1: Object was touched.
Debug: OPCODE: PUSHARGE
Debug: Param1: 0
Debug: OPCODE: PUSHSP
Debug: OPCODE: PUSHARGI
Debug: Param1: 8
Debug: OPCODE: ADD
Debug: Param1: 17
Debug: OPCODE: POPBP
Debug: OPCODE: CALLLIB_TWO_BYTE
Debug: Param1: 23
Debug: OPCODE: POP
Debug: OPCODE: RETURN
Debug: Last OPCODE was return command. Code chunk execution complete.
Debug: CLR:event_touch_start:il.BeginCatchBlock(typeof(Exception));
Debug: CLR:event_touch_start:il.Emit(OpCodes.Ldstr...
Debug: CLR:event_touch_start:il.Emit(OpCodes.Call...
Debug: CLR:event_touch_start:il.Emit(OpCodes.Callvirt...
Debug: CLR:event_touch_start:il.Emit(OpCodes.Call...
Starting CLR dynamic execution of: event_state_entry
Hello, Tedd!

.Net CLR Runtime

2007-07-12T10:44:00.000+02:00

The first thing I did was to see if .Net CLR looked anything like LSL ByteCode at all. As expected it did. It should be possible to convert between the two. But because we need to have scripts cross sims, the standard way of doing things might not suffice.

So, next question. How does one create CLR? Well, there are many ways. The ones I looked at as potential candidates was 1) creating an CLR code file and compiling and 2) creating CLR code run-time.
I chose to go with the latter one.

Here is the important part of my sample Hello World! run-time generated CLR program. The biggest Hello World! program I've ever made.

private void GenerateIL(ILGenerator il, OpenSimAPI.SimWorldAPI WorldAPI)
{
/*
 * TRY
 */
il.BeginExceptionBlock();
// Push "Hello World!" string to stack
il.Emit(OpCodes.Ldstr, "Hello World!");
Push Console.WriteLine command to stack ... Console.WriteLine("Hello World!");
il.Emit(OpCodes.Call, typeof(Console).GetMethod("WriteLine", new Type[] { typeof(string) }));

il.ThrowException(typeof(NotSupportedException));

/*
 * CATCH
 */
il.BeginCatchBlock(typeof(Exception));
// Push "Hello World!" string to stack
il.Emit(OpCodes.Ldstr, "Something went wrong: ");
//call void [mscorlib]System.Console::WriteLine(string)
il.Emit(OpCodes.Call, typeof(Console).GetMethod("Write", new Type[] { typeof(string) }));
//callvirt instance string [mscorlib]System.Exception::get_Message()
il.Emit(OpCodes.Callvirt, typeof(Exception).GetMethod("get_Message"));
//call void [mscorlib]System.Console::WriteLine(string)
il.Emit(OpCodes.Call,
typeof(Console).GetMethod("WriteLine", new Type[] { typeof(string) }));
/*
 * END TRY
 */
il.EndExceptionBlock();

// Push "Return from current method, with return value if present" to stack
il.Emit(OpCodes.Ret);

}

LSO / LSL ByteCode

2007-07-12T10:32:00.000+02:00

The task I've taken on me is to create script support for the OpenSim server.

This involves reading the compiled LSL script (LSL ByteCode, machine code for those who haven't heard of ByteCode) and executing it somehow.

The plan is to convert LSL ByteCode into .Net ByteCode, passing it through .Net CLR (MSIL/.Net Assembly Language) and having the JIT (Just In Time) compiler to optimizations.

The result should be blazing fast execution (operating system specific native code) operating within the .Net Virtual Machine.

Security will be through .Net's AppDomain restrictions. Powerfull enough in itself so we don't have to worry about anything security-related.

LSL Builtin function implementation will be done through a C# middle layer, since it is mostly specific to LSL. This will also allow other developers to implement the functions while I continue hacking on the rest of LSL-support.

There are two tricky parts to this project.

Running multiple scripts simultaneous
Allowing scripts to cross regions while running, thus moving script to another server.

Running multiple scripts simultaneous can not be done by assigning threads to scripts, because the number of scripts will be too high. Therefore user space threading (fibres) has to be implemented. The plan is to do this by inserting breakpoints into the CLR code that allows script execution to jump out and continue execution at a later point.

Solving multitasking in that manner also makes it easier solving the problem of allowing scripts to cross regions while running, since their current execution state can be saved to file and resumed later. The trick is to make sure that stack/heap/all memory is saved in the same manner.

So, let's see how it goes... :)

Second Life

2007-07-12T10:29:00.000+02:00

Second Life is a virtual world created by you and me. It's like Web 2.0, only World 2.0. Meaning you can make, do and find anything there. And it is growing really fast.

Many many many years ago, back in the late 90's when I was still young, I played around with the thought of creating such world. I had it all figured out, but 3D programming has never been of much interest to me. I made some fun sin/cos algorithms and made my own name spin in 3D back in QBasic and DOS days. Played around with DirectX and WildTangent. But never anything serious. And now someone has made it (SL), even better than I imagined. Happy happy! :)

Safe to say, I like it. But its still very limited. I can make a few objects and a few scripts. But I want to make animals with artificial intelligence, even a full ecosystem. This would require a freedom LSL and Lindens servers can not offer at this time.

I played around a bit with libsecondlife to control my character, and sure it was fun enough, but not what I was looking for.

So I found out someone had started writing OpenSim. An Open Source implementation of Second Life server. And less than 2 weeks ago I joined the project.

My Character

2007-07-12T10:19:00.001+02:00

I'm Tedd Maa in Second Life.
One of Tedd Hansens cyberegos.

I joined Second Life aprox 1 year ago, and have been looking at what people have built, done some scripting and made my own bot using libsecondlife.

Hello World!

2007-07-12T10:14:00.000+02:00

Hello World!