Scripting Extension: Difference between revisions

From OpenMW Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 30: Line 30:
                 {
                 {
                     std::string scriptfile = runtime.getStringLiteral (runtime[0].mInteger);
                     std::string scriptfile = runtime.getStringLiteral (runtime[0].mInteger);
                    runtime.pop()
                     //create another runtime that belongs to this OpStartExternalScript class so we'll have access in future?
                     //create another runtime that belongs to this OpStartExternalScript class so we'll have access in future?
                     Py_Initialize();
                     Py_Initialize();

Revision as of 19:22, 7 January 2015

These are notes on the scripting extension work, see: First described for python (SWIG): https://forum.openmw.org/viewtopic.php?f=6&t=617&start=60#p29966

LUA implementation: https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30090 https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30093

current ideas: I think a post-1.0 goal, of mine at least, will still be to see if I can get an easy generic scripting interface.

Overview:

  • It will be a seperate fork from openmw, as I think a unified scripting language and zini's enhancements to the existing implementation would be best for the community. It's generally better to improve one implementation rather than create several new ones. If, in the future, such extension of scripting is found desireable, then there is no problem incorporating it into openmw.
  • There are some things that python/other languages can do better and can be implemented faster than writing a new scripting engine from scratch. For example, you can use all of python's existing modules and capabillities that could enable things like dynamic web content/interaction in games. Some people are already used to lua and python scripting, they could potentially write more complicated scripts or more easily integrate scripts. For discussion on python versus lua, I found this very helpful:http://lua-users.org/wiki/LuaVersusPython -- I think both could be usefull in different cases.
  • Existing scripting IDEs can be used, most scripting language already has built-in verbose debugging. It's possible to generate embedded consoles for testing scripts on the fly. Scripts can be edited and recompiled on the fly, in game.
  • Security will be an issue (running untrusted scripts), but it something people already deal with in the modding community (e.g, Blender Python Scripting)Security. Concerns can be addressed at a minimum by making users opt in to using python, and can be extended to only allow running scripts signed by a trusted certificate/key.

Implementation:

  • Create a new opcode and scripting command for "StartExternalScript". This new opcode will have execute() function that take a runtime with the name of a script file, create an embedded interpretter for whatever language, and execute the script in the embedded interpretter.
namespace MWScript
{
    namespace External
    {
        class OpStartExternalScript : public Interpreter::Opcode0
        {
            public:
                virtual void execute (Interpreter::Runtime& runtime)
                {
                    std::string scriptfile = runtime.getStringLiteral (runtime[0].mInteger);
                    runtime.pop()
                    //create another runtime that belongs to this OpStartExternalScript class so we'll have access in future?
                    Py_Initialize();
                    FILE *file_1 = fopen(scriptfile.c_str(),"r");
                    PyRun_SimpleFileEx(file_1,scriptfile.c_str(),1);
                    Py_Finalize();
                }
        }

        void installOpcodes (Interpreter::Interpreter& interpreter)
        {
            interpreter.installSegment5 (Compiler::External::opcodeStartExternalScript, new OpStartExternalScript);
        }
    }
}


Will also need to define the actual int opcode in compiler::external, registerextensions, etc.

[question: should we create a virtual machine for every script, or just once and save a pristine environment to pass to it?]

Users can then create normal morrowind scripts such as:

StartExternalScript, "somepythonscript.py" or target->StartExternalScript, "somepythonscript.py"


  • The scripting language will have access to all the MWScript functions via binding functions. Binding functions can be generated programatically but calling "Compiler:registerExtensions (extension)" and generating c++ code from the resulting information on each function and instruction. A new tool can be made that links against libcomponents.a and generates binding functions and headers.

example tool:

Compiler::Extensions& extensions;
Compiler::registerExtensions(extensions);
std::vector<std::string>& keywords;
extensions.listkeywords(keywords);

Compiler::ScriptReturn& returnType=0;
Compiler::ScriptArgs& argumentType=0;
bool& explicitReference=0;

for each keyword
{
 std::string cppcode="";
 std::string return_type="";
 std::string arguments="";
 if (extension:isFunction (keyword, returnType, argumentType, explicitReference)
 {
   switch(returnType)
   {
     case "f":
      return_type+="Interpretter::Type_Float ";
      return_command = "return runtime[0].mFloat"
      break;
     case "S":
      return_type+="std::string ";
      return_command = "return runtime.getStringLiteral(runtime[0].mInteger)"
      break;
     case "l":
      return_type+="Interpretter::Type_Integer ";
      return_command = "return runtime[0].mInteger"
      break;
     and then the empty case:
      return_type+="void ";
      return_command = "return"
      break;
    }
   then parse all the argument types, for example fSl/l can be made into
     arguments = "Interpretter::Type_Float arg0, std::string arg1, Interpretter::Type_Integer arg2, Interpretter::Type_integer arg3={SOMEFLAG}"
     required_pushes=3;
     optional _push=1;
   cppcode = returntype+keyword+"("+arguments+")";
   you can also overload for explicit functions
     arguments = "std:string explicit, Interpretter::Type_Float arg0, std::string arg1, Interpretter::Type_Integer arg2, Interpretter::Type_integer arg3={SOMEFLAG}"
     required_pushes=4;
     optional _push=1;
   if first argument is string, it is explicit. Since we're using default values for optional arguments, could be some problems finding the right function though.

   and you can use cppcode for declaration, output to header file and to implementation code.
   Next generate the code to implement keyword's opcode function
   that was declared in cppcode. output the following to the implementation code
    1) Get the runtime that belongs to this external script, clear stack
    2) push required parameters on to runtime stack (first push explicitID if it is explicit)
    3) if optional variables are not {SOMEFLAG}, push them on the stack
    4) run opcode.execute(runtime)
    5) then add return_command to the output implementation code

  and then do the same with
  if (extension:Instruction (keyword, argumentType, explicitReference){...}
 }
}

you should now have declarations and the binding functions, which you can now parse with swig and build as part of the _openmw.so module.

  • Binding functions will be made to know the opcode, configure a runtime object using the script manager's current context, and push the variables passed to it in order on to the runtime stack. It will then run the appropriate opcode execute function. The function will be overloaded to allow for both explicit and implicit as well has have default values for optional parameters. See above tool that can generate those functions.

player->SomeCommand, 10, "somestring" would be somecommand("player", 10, "somestring")

SomeCommand, 10, "somestring" would be somecommand(10, "somestring")

function returns will be generated programatically as well based on in encoding


Outstanding questions: [*] How to handle saving variables? Between frames, variables can be saved by saving the environment. That also allows us to possibly think of creating each running script as a seperate object in python and make the interpretter environment persistent. Then add the variables to save upon exist (and load upon initiation) through a second binding function. Or we can create multiple independent environments. [*] How fast will this be? i'm guessing slower than the MWScript system, but will it be prohibitively slower?