Idling

I guess its that time of year again when I write my blog post...

It's been a while... :) I've been busy with RL work. Sadly that work doesn't involve OpenSim just yet. But now I'm back and I thought I'd put some words here.

I've picket up where I left off on ScriptServer. Not so much to report yet. TCP server/client implementation is ready, binary protocol and bindings are not. Server executable is ready, plugin module is ready. So what remains is mainly binary protocol and binding implementations.

Depending on how much free time I have this shouldn't take too long.

Script engine implementation

I just wrote this contribution to AWG: http://wiki.secondlife.com/wiki/Tedds_stand-alone_script_engine

It explains how and why on my opinions on the future script engine implementation.

Communication

Things are a bit quiet right now. But there are things happening behind the scenes.

Some notes on communication between region and script engine.

Region will be the initiator. That is, the one that contacts script server. If an object containing a script moves from one region to another then the next region will contact the script server.

The communication protocol will be a custom binary protocol. As far as I know there are no RPC protocol designed for high volume async function calls. This will allow a continous stream of function calls in both directions. Any reply on commands will be done by attaching an reply ID to the function call package, a reply package refering this ID containing reply data will be sent back.

Scripts will be blocked while waiting for reply, but not while waiting for their command to be sent.

Currently TCP client/server has been created, but the binary protocol is still in design stage.

A peek on new architecture

This is where my planning is at now.
It was very complex, and after redrawing it a few times I neded up at this.
Still not 100%, but getting there...

Major rewrite number 2

I have been doing some thinking on the ScriptEngine. Actually a whole lot of thinking, and planning. Lots of alien squid-like drawings in my notebook.

Trying to run through the process: Idea, required elements, architecture and finally simplification.

The idea is simply: Moving ScriptEngine to a separate daemon, independent from the region.
This has several big advantages and of course some drawbacks.

• Script does not need to be moved between regions when object it is inside does.
• Scripts can run on separate servers and even clusters of servers, offloading the main region server.
• Script source (and binary) can be secured by running it from a trusted server. Even makes it possible with script rental. You can run your own ScriptEngine.
• We can add the option to move script between regions as usual to avoid network lag on time-critical scripts.
• It is easier to write alternative ScriptEngines, developers are not restricted to .Net.
• ScriptEngines can be more complex and allow bigger and more complex scripts than the region would by default.
• Regions can be more secure as they do not have to execute scripts locally.

Disadvantage:

• Each region may have to speak to many different scriptengines.
• Network latency will affect script performance.
• Script reliability is much lower.
• Various problems related to authentication.

Of course all the work done with implementing LSL commands so far will survive. That is independent of this rewrite.

I'll be back with more as it unfolds.

Perfect code

The development process of a good program may often seem strange for an outsider. One can expect the development process to consist of planning, development and testing. A simple linear way to get from design to final product. In most commercial scenarios this is also how its done. Forcing an application from idea through design, programming and testing to the final product. And this is right, because in almost every case a program is design to do a specific task. There are rarely and demands on how the code looks or how secure it is. The application will be serving 1, 2 or 50 people for 2-3 years with little modifications once its has been developed.

However when a programmer who loves to code is sitting at home developing something the process is very different. Programs are rarely finished, they are often only 70-80% done before scrapped and a new one is started. Debugging has been done during development, and the program is likely to contain lots of bugs. Because the program was not written for the purpose of being used. It was a practice... A practice in architecture, algorithms, API's and objects. The next time the a program is written the past experience is put into that program, and its slightly better. It's all about understanding the architectures and algorithms. Therefore, in search for the "perfect code", you find developers rewriting their applications - time after time. Sometimes it's major restructuring, sometimes major rewrites. Learning how to plan for change, so that you don't need to plan.

Do I need to say more? nah.

Pure C# support

ScriptEngine supports both LSL and C# within the same script. Mainly this means that you can write LSL, but when you want you can use C# code. For example instead of using llToLower(mystring) you can use mystring.ToLower(), instead of llStringLength(mystring) you can use mystring.Length().

But to make sure that we support pure C# code, I've added support for forcing the compiler to not use LSL preprocessor. This means skipping directly to compile without going through LSL to C# converter.

Any script that starts with the 4 letters "//C#" or //c#" will be compiled as C# directly. This ensures that we have 100% C# support.

There are a few rules you need to follow. Mainly:

• Any LSL event needs to be renamed to <STATE>_event_<EVENTNAME>.
  For example "start_entry()" in default state will be "public void default_event_state_entry()"
• LSL events needs to be public, of course.
• Script needs to start with "//C#" without the quotes and without space in front of it. These 4 characters needs to be the first thing inside the script!
• You still have to use ll-commands for stuff that native C# doesn't support. But don't worry - the ll-commands are just the name, you are actually writing in C#. ;)

If you put a LSL script into an object in OpenSim, the C# code of that script will be debug-logged to the "bin\ScriptEngine\"-folder. So if you have any trouble, just write up a LSL code, put it into an object and you have a working C# script that does the same thing.

Untested example:

//C#

namespace SecondLife
{

   public class Script : OpenSim.Region.ScriptEngine.DotNetEngine.Compiler.LSL.LSL_BaseClass
   {

      public Script() { }

      // Your script goes here!
      int touch_count = 0;
      int tick_count = 0;
      int objnum = 1;

      public void default_event_state_entry()
      {
         llSay(0, "Hello, Avatar! Starting timer...");
         llSetTimerEvent(10);
      }

      public void default_event_touch_start(int total_number)
      {
         touch_count++;
         llSay(0, "Object was touched. Touch count: " + touch_count);
      }

      public void default_event_timer()
      {
         tick_count++;
         llSay(0, "Timer count: " + tick_count);
      }
   }
}

Starting my own company

Hi

I'm currently in a period of less activity on the development front. Very sorry about that.
At the end of last month I quit my job and started my own company as an independent consultant, and with this comes a lot of work setting up, getting customers, etc...

The good news is that once this is through I will be back for full. The plan is that my company will support this project through my development effort, so there will be time set of specially for OpenSim. But the time I can put into it will of course vary a bit depending on what projects I'm on.

But I'm looking very much forward to starting back up on OpenSim. The plan is tomorrow (monday) or tuseday, depending on finalization of current project. But my mind has not been completely absent, I've been thinking about the ScriptEngine allthough I haven't had time to do any coding.

This is what I want to fix next:

• Look into how to move ScriptEngine from Region to Sim-level with MW, changing object and script identifiers to LLUUID.
  This way a script can keep running even though it crosses local regions - a great benefit to both the script itself and the servers.
• Document how so anyone who wants to take on the mission of creating llHttpRequest can do so. (this is one of those long commands that needs cleaning up if a script is removed)
• Serialize/Deserialize code in AppDomains so we can move running scripts (that are inbetween executions) and keep their state when stopping/starting server.
• Make scripts work in Grid-mode.

Also I have been playing with the idea of restructuring ScriptEngine so that it can easily be used in other projects too. But we'll have to wait and see...

WIKI updated with info on scripting

I've updated the OpenSim Wiki with:

• General info about scripting in OpenSim
• llFunction implementation status
• And some information about the DotNetScriptEngine internals

Small stresstest

114 scripts loaded. 1 script per AppDomain. Most scripts are alone inside a prim, except the big box over there that has something like 10 scripts in it (it is bigger so I could fit more scripts into it).







My computer is a 2,8GHz Prescott with 2GB RAM and Windows Vista Ultimate.
60 of the scripts run the timer event every 1 second. 50 run it every 5 seconds.

Basically this tells us that ScriptEngine uses a bit too much memory. This is expected. CPU usage while running 114 events every x seconds is almost nothing.

Increase from 38MB to 76MB RAM when creating objects and scripts, so ~10MB for 25 scripts+objects.

The scripts don't do much work right now, but if they did the speed would still be high as they are compiled to CPU native code.

Results after running for 9 hours

OpenSim did not crash or eat any more CPU or memory over time. However the Second Life viewer did crash, so I had to log in again... Funny ;)