Charles Krinkle has helped in adding interface and necessary empty prototype implementations of base class for the compiled script.
Almost all functions has been implemented (they contain no code yet though). This helps me test the compiler on all kinds of scripts without getting errors.
Thanks dude! :)
Tuesday, August 14, 2007
New status after Monday (and a milestone)
ScriptEngine has now been implemented!
We have a default script that is placed in all objects. We have a "touch_start" event. And we have llSay() with output to server console.
Once inventory is working we can bind user made scripts to user made objects. So now we can start implementing all 350+ builtin ll-functions.
We will be able to test the functions in-world as they are being implemented. Just modify the script "Default.lsl" under "bin\ScriptEngines\" folder to use the new command you are implementing.
There may still be problems compiling some scripts. Please let me know about these problems so I can fix them!
Next step for me is to implement error handling around ScriptEngine.
We have a default script that is placed in all objects. We have a "touch_start" event. And we have llSay() with output to server console.
Once inventory is working we can bind user made scripts to user made objects. So now we can start implementing all 350+ builtin ll-functions.
We will be able to test the functions in-world as they are being implemented. Just modify the script "Default.lsl" under "bin\ScriptEngines\" folder to use the new command you are implementing.
There may still be problems compiling some scripts. Please let me know about these problems so I can fix them!
Next step for me is to implement error handling around ScriptEngine.
Sunday, August 12, 2007
Status of LSL compiler
Status update
LSL->C# Translator is close to complete. Needs bigger/more scripts to bughunt.
Compiler is capable of compiling C# to .Net (invokes LSL->C# trans if needed).
Script Manager is capable of loading and initializing script .Net Assembly.
Missing:
LSL BuiltIn command implementation.
Event implementation.
LSL->C# Translator is close to complete. Needs bigger/more scripts to bughunt.
Compiler is capable of compiling C# to .Net (invokes LSL->C# trans if needed).
Script Manager is capable of loading and initializing script .Net Assembly.
Missing:
LSL BuiltIn command implementation.
Event implementation.
How to contribute?
OpenSim.Region.ScriptEngine.DotNetEngine.Compiler.LSL.LSL_BaseClass.cs contains a s*load of empty functions. These are all the LSL-functions. And all needs to be implemented. Implement most important/most used first.
There is a "World" object available (scene).
Example of implementation:
There is a "World" object available (scene).
Example of implementation:
public void llSay(Int channelID, string text)
{
World.Something.Somewhere.SendMessageToEveryoneAround(channelID, text);
}
LSL to C#
To compile LSL code I figured the best way would be to first translate it into C#.
Since LL will be supporting both LSL and C# we can use the same translator on a web page or similar to help people who know LSL to understand C#.
Also it is much simpler for other developers on OpenSim to contribute to the code when its C# instead of MSIL.
The translator uses some regex and a couple of byte-by-byte passes over the code. Written in less than a day, no big magic involved.
In-memory compile to assembly (that will be cached for later reuse).
Assembly is dynamically loaded in same way as LSO.
The script inherits from a LSL_BaseClass.cs that contains all the builtin-functions like llSay(), llWhisper, etc.
Since LL will be supporting both LSL and C# we can use the same translator on a web page or similar to help people who know LSL to understand C#.
Also it is much simpler for other developers on OpenSim to contribute to the code when its C# instead of MSIL.
The translator uses some regex and a couple of byte-by-byte passes over the code. Written in less than a day, no big magic involved.
In-memory compile to assembly (that will be cached for later reuse).
Assembly is dynamically loaded in same way as LSO.
The script inherits from a LSL_BaseClass.cs that contains all the builtin-functions like llSay(), llWhisper, etc.
Changing tactics
After a meeting with Babbage Linden in Zero Lindens office hours on thursday I realized that with the new Mono/.Net compiler they are implementing we will have to make a few adjustments.
It was earlier assume that compilation would be done at client of both LSL and LSO, therefore I focused on making a LSO (LSL virtual machine) ByteCode to .Net CIL (Assembly language) translator.
But because of some security concerns in the .Net ByteCode LL has chosen to do compilation on server. LSL or C# code is sent to server where it is compiled and put into an object. The compiled code is never sent from the viewer.
Therefore I am now writing a LSL compiler that will run on server.
While waiting for Linden to update their client (probably 0,5-1 years) we will use the text-copy of LSL script sent by viewer and compile that. Meaning we are not using the LSO at all.
So basically we are scrapping the LSO compiler, and going for a LSL compiler. There are many clear advantages to this, so I'm quite happy about it.
It was earlier assume that compilation would be done at client of both LSL and LSO, therefore I focused on making a LSO (LSL virtual machine) ByteCode to .Net CIL (Assembly language) translator.
But because of some security concerns in the .Net ByteCode LL has chosen to do compilation on server. LSL or C# code is sent to server where it is compiled and put into an object. The compiled code is never sent from the viewer.
Therefore I am now writing a LSL compiler that will run on server.
While waiting for Linden to update their client (probably 0,5-1 years) we will use the text-copy of LSL script sent by viewer and compile that. Meaning we are not using the LSO at all.
So basically we are scrapping the LSO compiler, and going for a LSL compiler. There are many clear advantages to this, so I'm quite happy about it.
Sunday, August 5, 2007
Small milestone reach today
Application is now able to:
- Create Script Engine (SE)
- SE hooks up to a temporary debug event in Test App
- Test App asks SE to load LSO-file
- SE compiles LSO-file into .Net Assembly
- SE loads .Net Assembly
- SE Initializes the .Net Assembly and hands over a "LSL BultIn Commands" object (this object is the scripts command-object)
- Script Manager adds Script to an Object[].Script[] structure
- By pressing a button in Test App an event is sent to SE
- SE receives event and queues it to correct ObjectID + ScriptID
- Separate thread dequeues event and sends it to Script Manager
- Script Manager executes function
- llSay() is displayed to debug console
Status on initial SE implementation
With the model described in the two prior postings we will have simple scripts support up and running fairly quickly.
Other developers will be able to extend script support while I continue working on Compiler and ScriptEngine. Both incoming events and outgoing LSL builtin commands needs extending.
We will lack script yield inserting. This is where the preparation of .dll for microthreading will happen.
We will have a simplified version of Event Queue Manager that will use a custom threadpool to execute events and brutally kill any long-running events.
We will havea simplified version of AppDomain handling that basically just loads all scripts into one AppDomain with no code security.
We will have a basic LSL Compiler that can compile the simplest LSL scripts, but that others can extend command support for.
In short, the scripts should be able to llSay() during rez. But can easily be extended by anyone to support the commands we need. The basic structure is in place, and we can build upon that bit by bit.
Other developers will be able to extend script support while I continue working on Compiler and ScriptEngine. Both incoming events and outgoing LSL builtin commands needs extending.
We will lack script yield inserting. This is where the preparation of .dll for microthreading will happen.
We will have a simplified version of Event Queue Manager that will use a custom threadpool to execute events and brutally kill any long-running events.
We will havea simplified version of AppDomain handling that basically just loads all scripts into one AppDomain with no code security.
We will have a basic LSL Compiler that can compile the simplest LSL scripts, but that others can extend command support for.
In short, the scripts should be able to llSay() during rez. But can easily be extended by anyone to support the commands we need. The basic structure is in place, and we can build upon that bit by bit.
ScriptEngine - Events
The builtin commands for SE is simple enough. But incoming events could need some more planning.
SE will have a set of events that is fired based on stuff that happens in OpenSim (EventHandler). This EventHandler will translate the event into a LSL-compatible event, including picking up the required parameters.
The EventHandler needs to know what object, and possibly what script, the event is for. Then it will queue this event for the correct script(s) (multiple scripts may subscribe to the same event in an object).
A Event Queue Manager will process the event queue under a separate thread and execute the correct script functions with the correct parameters. By using a separate thread OpenSims event thread is free to return immediately.
What scripts subscribe to what events is determined during script load. Either by dynamically identifying function names, or by having the script hook itself up during load. The functions will be stored by reference to avoid using expensive Reflection-calls for every script event.
One possiblity for extra speed is to dynamically (using CIL) create a call object in same AppDomain as script is located that does some of Event Queue Managers work.
SE will have a set of events that is fired based on stuff that happens in OpenSim (EventHandler). This EventHandler will translate the event into a LSL-compatible event, including picking up the required parameters.
The EventHandler needs to know what object, and possibly what script, the event is for. Then it will queue this event for the correct script(s) (multiple scripts may subscribe to the same event in an object).
A Event Queue Manager will process the event queue under a separate thread and execute the correct script functions with the correct parameters. By using a separate thread OpenSims event thread is free to return immediately.
What scripts subscribe to what events is determined during script load. Either by dynamically identifying function names, or by having the script hook itself up during load. The functions will be stored by reference to avoid using expensive Reflection-calls for every script event.
One possiblity for extra speed is to dynamically (using CIL) create a call object in same AppDomain as script is located that does some of Event Queue Managers work.
Saturday, August 4, 2007
ScriptEngine
The time has come to start implementing script support into OpenSim.
ScriptEngine (SE)
OpenSim will have an interface for SE. It will load SE dynamically using reflection. SE will be in separate .dll Assembly.
During load SE will hook itself up to the necessary events in OpenSim.
This includes: Object rez (look for active scripts), object derez and all events relating to objects (touch, sit, etc).
SE might also during startup need to iterate through existing objects to see what scripts it needs to start. This depends on at what point SE is loaded.
All access to OpenSim from scripts will go through SE.
Compiler
When SE starts a script it will determine script type. If script type is LSO (LSL ByteCode) the script will be compiled into .Net Assembly .dll before load. Compiler is a separate .dll Assembly, but hardcoded into SE.
Loading scripts
* If script is LSO, compile to .Net Assembly
* Modify .Net Assembly to support microthreading (will be done later in project)
* Find available AppDomain for the script (currently only default appdomain)
* Load script into AppDomain
* Create a private instance of BuiltIn commands for the script
* Create object of script handing it the private BuiltIn command instance
Some BuiltIn functions will require information about who is executing them, for example for object permissions or feedback events. Because .Net Assembly can not be trusted (it is uploaded by users) ScriptEngine will create a private instance of BuiltIn commands. This instance will contain and pass on the necessary ID's to OpenSim - as well as work as a translator between LSL-functions and OpenSim API.
Operation
A script instance is made up of ScriptID and ObjectID. This should always result in an unique combination for any script.
ScriptEngine will receive events from OpenSim and forward the events to the required scripts. During this forward it will also enforce microthreading.
Current Status
Class OpenSim.ScriptEngine.DotNetEngine.ScriptEngine has basic framework and is able to load and execute scripts. Only missing event hookups and actual implementation into OpenSim.
ScriptEngine (SE)
OpenSim will have an interface for SE. It will load SE dynamically using reflection. SE will be in separate .dll Assembly.
During load SE will hook itself up to the necessary events in OpenSim.
This includes: Object rez (look for active scripts), object derez and all events relating to objects (touch, sit, etc).
SE might also during startup need to iterate through existing objects to see what scripts it needs to start. This depends on at what point SE is loaded.
All access to OpenSim from scripts will go through SE.
Compiler
When SE starts a script it will determine script type. If script type is LSO (LSL ByteCode) the script will be compiled into .Net Assembly .dll before load. Compiler is a separate .dll Assembly, but hardcoded into SE.
Loading scripts
* If script is LSO, compile to .Net Assembly
* Modify .Net Assembly to support microthreading (will be done later in project)
* Find available AppDomain for the script (currently only default appdomain)
* Load script into AppDomain
* Create a private instance of BuiltIn commands for the script
* Create object of script handing it the private BuiltIn command instance
Some BuiltIn functions will require information about who is executing them, for example for object permissions or feedback events. Because .Net Assembly can not be trusted (it is uploaded by users) ScriptEngine will create a private instance of BuiltIn commands. This instance will contain and pass on the necessary ID's to OpenSim - as well as work as a translator between LSL-functions and OpenSim API.
Operation
A script instance is made up of ScriptID and ObjectID. This should always result in an unique combination for any script.
ScriptEngine will receive events from OpenSim and forward the events to the required scripts. During this forward it will also enforce microthreading.
Current Status
Class OpenSim.ScriptEngine.DotNetEngine.ScriptEngine has basic framework and is able to load and execute scripts. Only missing event hookups and actual implementation into OpenSim.
Subscribe to:
Posts (Atom)