WIKI updated with info on scripting

I've updated the OpenSim Wiki with:

• General info about scripting in OpenSim
• llFunction implementation status
• And some information about the DotNetScriptEngine internals

Small stresstest

114 scripts loaded. 1 script per AppDomain. Most scripts are alone inside a prim, except the big box over there that has something like 10 scripts in it (it is bigger so I could fit more scripts into it).







My computer is a 2,8GHz Prescott with 2GB RAM and Windows Vista Ultimate.
60 of the scripts run the timer event every 1 second. 50 run it every 5 seconds.

Basically this tells us that ScriptEngine uses a bit too much memory. This is expected. CPU usage while running 114 events every x seconds is almost nothing.

Increase from 38MB to 76MB RAM when creating objects and scripts, so ~10MB for 25 scripts+objects.

The scripts don't do much work right now, but if they did the speed would still be high as they are compiled to CPU native code.

Results after running for 9 hours

OpenSim did not crash or eat any more CPU or memory over time. However the Second Life viewer did crash, so I had to log in again... Funny ;)

So where are we now?

[Updated Monday 13:45 GMT+1]
[Updated Wednesday 21:45 GMT+1]
[Updated Saturday 22:45 GMT+1]

As always its important with a summary for those who don't want to read techie details.

Here is current status of LSL script support:
+ We have LSL script support.
+ We can run hundreds of scripts simlutaneously, probably not thousands (needs testing). We have confirmed 50 simultaneous scripts without any sign of trouble.
+ We can utilize multicore CPU's for running scripts.
+ We can to some degree unload scripts to free up memory. (if we use 1 AppDomain per script)
+ Events from server are put into a queue, so server threads are released imediately.
+ Some llFunctions are implemented.
+ ScriptEngine is more or less ready to bind individual scripts to individual prims.
+ We have implemented touch_start event when someone touches a prim.
+ We have implemented llSetTimerEvent.
+ Script compile errors are displayed in-world.
+ We can put scripts into prims.

- Can not run too many scripts that contain infinite loops or sleeps to long.
- No memory/CPU sharing/limitation/throttling on scripts.
- Almost no security implemented on scripts.
- Can not move script between regions.
- Missing 300+ llFunctions.
- Missing most events.

There is room for making scripts run 2-5 times faster, if not more. And same in memory usage. This is because of the development process.
"Get it working, remove bugs, make it fast."


There is much data related to each script. I suspect if we load too many scripts the working set (WSClock) will be too high. Meaning there is too much data to process, so the CPU and memory will be more occupied with looking at what to do than actually doing it.

Introduction of a new class

Some LL-commands takes longer to return. These often return with a event. For example llSetTimerEvent(10) will return a timer() event every 10 seconds. llHttpRequest() will return when it receives answer from HTTP-server.

To handle these types of requests a new class has been added as a subclass of ScriptEngine. The class is named LSLLongCmdHandler.cs. This class will contain handlers for adding/removing requests. Adding when script executes a command, removing when command is not complete yet but script is being moved to another region or otherwise removed.

The class has a dedicated thread that will be working the different services, for example async HTTP requests and queueing of timer events at the correct intervals.
LSL_BuiltIn_Commands.cs accesses this class through ScriptEngine.


Example from LSL_BuiltIn_Commands.cs:

public void llSetTimerEvent(double sec)
{
  // Setting timer repeat
  m_ScriptEngine.myLSLLongCmdHandler.SetTimerEvent(m_localID, m_itemID, sec);
}

Speed

I've added support for multithreading execution of script events. Only one object will be processed at the time. This will benefit from MultiCore CPU's. Though it makes each call more expensive because we have to keep track of mutexed objects.

Also added caching of event binding (function call) so we only use Reflection on first execution of a given event within a given state.

Changed sleep time of inactive threads from 200ms to 50ms, so they are more responsive.

AppDomains

.Net does not allow unloading of assemblies. If you want to unload an assembly you have to unload the whole AppDomain.

I've created AppDomainManager that handles creating AppDomain, finding a free AppDomain, loading scripts into it, and unloading AppDomains with no active scripts in them.

Its purpose is to create AppDomains and put X number of scripts into each of them. Later when we get microthreading support we will add a feature to move scripts between AppDomains, so if one or more AppDomains has only a few active scripts we can move these to a fresh AppDomain and unload the old one.

With the introduction of AppDomain there came a few changes to the source. Because of how communication between AppDomains (mainly between scripts AppDomain and the main AppDomain) works there has been made some changes.

First of all a new assembly called OpenSim.Region.ScriptEngine.Common.dll has been introduced. It contains interface for scripts and some common code for all AppDomains to execute events inside scripts.

Secondly the LSL BuiltIns (llFunctions) has been moved to a separate file (LSL_BuiltIn_Commands.cs) and are no longer part of the LSL_BaseClass.cs.

Also OpenSim.Region.ScriptEngine.Common.LSL_Types that contains custom Vector and Rotation objects is now used throughout DotNetEngine/Compiler.

Each AppDomain has only 2 references, OpenSim.Region.ScriptEngine.Common.dll and OpenSim.Region.ScriptEngine.DotNetEngine.dll.

llSay implemented

Picture of llSay() implemented.

llFunction interface/empty prototypes

Charles Krinkle has helped in adding interface and necessary empty prototype implementations of base class for the compiled script.

Almost all functions has been implemented (they contain no code yet though). This helps me test the compiler on all kinds of scripts without getting errors.

Thanks dude! :)

New status after Monday (and a milestone)

ScriptEngine has now been implemented!

We have a default script that is placed in all objects. We have a "touch_start" event. And we have llSay() with output to server console.

Once inventory is working we can bind user made scripts to user made objects. So now we can start implementing all 350+ builtin ll-functions.

We will be able to test the functions in-world as they are being implemented. Just modify the script "Default.lsl" under "bin\ScriptEngines\" folder to use the new command you are implementing.

There may still be problems compiling some scripts. Please let me know about these problems so I can fix them!
Next step for me is to implement error handling around ScriptEngine.

Status of LSL compiler

Status update

LSL->C# Translator is close to complete. Needs bigger/more scripts to bughunt.
Compiler is capable of compiling C# to .Net (invokes LSL->C# trans if needed).
Script Manager is capable of loading and initializing script .Net Assembly.

Missing:
LSL BuiltIn command implementation.
Event implementation.