Scripting Extension: Difference between revisions

From OpenMW Wiki
Jump to navigation Jump to search
No edit summary
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Overview ==
These are notes and current status of extending openmw scripting through SWIG.
These are notes and current status of extending openmw scripting through SWIG.


STATUS: All extensions can be called from python scripts (both instructions and functions). Python scripts can be run in the console or attached to objections. Local/global variables do not work yet.
== Current Status ==
Python and Lua scripting is fully functional. You can call from the console, or can attach to objects/global scripts with the command <pre>StartExternalScript, "scriptname.py"</pre> or <pre>StartExternalScript, "scriptname.lua"</pre> This way you get access to all the locals and globals as if your script is a normal script. The Python/Lua script has to be in the same directory that other morrowind data/addons are in (one of the data directories).


These are notes on the scripting extension work, see:
== How to Install and Use ==
First described for python (SWIG):


https://forum.openmw.org/viewtopic.php?f=6&t=617&start=60#p29966
=== Install Prerequsites ===
'''Ubuntu Linux:'''<br />
(Should also work in other Linuxes that use the <samp>apt</samp> package manager.)
<pre>
$ sudo apt-get update
$ sudo apt-get install python3-dev swig liblua5.1-0-dev
</pre>
<!--Add cases for yum and other package managers; Homebrew and/or MacPorts for macOS; Cygwin for MS Windows.-->


LUA implementation:
=== Get the modified openmw binary ===


https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30090
==== Github ====
 
Command-line example for Unix-like operating systems:
https://forum.openmw.org/viewtopic.php?f=6&t=617&start=70#p30093
<pre>
 
How to use:
<pre>$ sudo apt-get install python3-dev swig
$ git clone https://github.com/maqifrnswa/openmw/tree/python-scripting  
$ git clone https://github.com/maqifrnswa/openmw/tree/python-scripting  
$ cd openmw/
$ cd openmw/
$ mkdir build && cd build
$ mkdir build && cd build
$ make -j
$ make -j
$ [make your python script, see example below]
$ PYTHONPATH=. ./openmw --start="Seyda Neen" --skip-menu
</pre>
</pre>
then in game, hit "~" to get the console, and run your script with the new command <pre>StartExternalScript, "test.py"</pre>
<!--Add instructions for Cygwin for MS Windows.-->


Example script (no need to import openmw, it will do it automatically):
==== Ubuntu PPA ====
A Utopic personal package archive (PPA) is available for Ubuntu (and related) Linux:
<pre>
<pre>
print("health: " + str((gethealth("player"))))
$ sudo apt-add-repository ppa:showard314/openmw-scripting
$ sudo apt-get install openmw
</pre>


print("hello i'm a criminal")
It will install the highest version of openmw in all the repositories you have enabled, so if you have the openmw PPA, the daily PPA, and this PPA turned on, it may download versions from unwanted PPAs as each gets updated. Therefore, to be safe, only enable one PPA in the Software Center Sources at a time. This version also comes with "testing.py" as seen in the Example below.
setpccrimelevel(100000000)


print("flying: " + str(getflying("player")))
=== Script Testing ===
setflying("player",1)
Make your Python or Lua script; see Python example below. The test the script in the console.  A command-line example for most Unix-like operating systems:
print("flying: " + str(getflying("player")))
<pre>
$ ./openmw --start="Seyda Neen" --skip-menu
</pre>
<!-- In macOS:  *DON'T NEED THIS UNTIL INSTALL INSTRUCTIONS ARE DONE.*
<pre>
/Applications/OpenMW.app/Contents/MacOS/openmw --start="Seyda Neen" --skip-menu
</pre>
-->
<!--Add example for Command Prompt in MS Windows.-->
Then in game, press <code>~</code> to get the Console, and run your script with the new command <code>StartExternalScript, "testing.py"</code> or <code>StartExternalScript, "testing.lua"</code>, or write a script/addon that calls <code>StartExternalScript</code>.
 
=== Python Scripts ===
OpenMW will import your python script as a module and run the <code>run()</code> method when you use the Morrowind command <code>StartExternalScript</code>.
 
==== Example Script====
Make sure you have a <code>run()</code> method, that is what will be called by OpenMW:
<pre>
#Example file: "testing.py"
#SomethingScript is a script in a .omwaddon file that you create that has a short variable named "hello" you can use for testing.
#It's just a demonstration of setting a local variable, you can comment it out if it causes problems.
from openmw import *
 
def run():
    print("health: " + str((gethealth("player"))))
    print("hello i'm a criminal")
    setpccrimelevel(100000000)
    print("flying: " + str(getflying("player")))
    setflying("player",1)
    print("flying: " + str(getflying("player")))
    print("random100: " + str(omwget("random100")))
    omwset("random100",42)
    print("random100: " + str(omwget("random100")))
    print("SomethingScript.hello: " + str(omwget("SomethingScript.hello")))
    omwset("SomethingScript.hello",42.0)
    print("SomethingScript.hello: " + str(omwget("SomethingScript.hello")))
    omwcall("MessageBox, \"This is a simple message\"")
</pre>
 
==== How to Write Python Scripts ====
All commands are lowercase and take the same arguments as before, even optional ones. Commands that can take a reference (commands with <code>-&gt;</code>) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use <code>self</code>.
 
Example 1:
<pre>
player->AddItem, "Gold_001", 200
</pre>
is now:
<pre>
additem("player","Gold_001",200)
</pre>
</pre>


how to write scripts:
All commands are lowercase and take the same arguments as before, even optional ones. Commands that can take a reference (commands with ->) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use "self". Example 1:
<pre>player->AddItem, "Gold_001", 200 </pre> is now <pre>additem("player","Gold_001",200)</pre>
Example 2:
Example 2:
<pre>AIWander, 0, 0, 0 </pre> is now <pre>aiwander("self",0,0,0)</pre>
<pre>
AIWander, 0, 0, 0
</pre> is now:
<pre>
aiwander("self",0,0,0)
</pre>
 
Example 3:
Example 3:
<pre>"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0</pre> is now <pre>aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)</pre>
<pre>
"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0
</pre>
is now:
<pre>
aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)
</pre>


Also, use
===== Getting and setting variables =====
<pre>omwset("localvariablename", value)
<pre>
omwset("localvariablename", value)
omwset("globalvariablename", value)
omwset("globalvariablename", value)
omwset("objectID.variablename", value)
omwset("objectID.variablename", value)
Line 52: Line 114:


omwget("localvariablename")
omwget("localvariablename")
etc.</pre>
[etc.]
instead of set and get to avoid name collisions with functions in python.
</pre>
Use <code>omwget</code> and <code>omwset</code> instead of <code>get</code> and <code>set</code> to avoid name collisions with built-in functions in Python.
 
===== Non-extension Functions and Instructions =====
All extensions are directly available for use, but game interpreter commands (like <code>MessageBox</code>) can be accessed with <code>omwcall()</code>:
<pre>
omwcall("MessageBox, \"This is a simple message\"")
</pre>
 
The argument to <code>omwcall</code> will be parsed as if it were a normal Morrowind script.
 
=== Lua Scripts ===
OpenMW will import your Lua script as a module and run the <code>run()</code> method when you use the Morrowind command <code>StartExternalScript</code>
 
==== Example Script ====
Make sure you have a <code>run()</code> method; that is what will be called by OpenMW:
<pre>
#Example file: "testing.lua"
#SomethingScript is a script in a .omwaddon file that you create that has a short variable named "hello" you can use for testing.
#It's just a demonstration of setting a local variable; you can comment it out if it causes problems.
 
local testing = {}
 
function testing.run()
    print("health: "..omw.gethealth("player"))
    print("hello i'm a criminal")
    omw.setpccrimelevel(100000000)
    print("flying: "..omw.getflying("player"))
    omw.setflying("player",1)
    print("flying: "..omw.getflying("player"))
    print("random100: "..omw.omwget("random100"))
    omw.omwset("random100",42)
    print("random100: "..omw.omwget("random100"))
    print("SomethingScript.hello: "..omw.omwget("SomethingScript.hello"))
    omw.omwset("SomethingScript.hello",42.0)
    print("SomethingScript.hello: "..omw.omwget("SomethingScript.hello"))
    omw.omwcall("MessageBox, \"This is a simple message\"")
end
 
return testing
</pre>
 
==== How to Write Lua Scripts ====
All commands are in the <code>omw</code> namespace, are lowercase, and take the same arguments as before, even optional ones. Commands that can take a reference (commands with <code>-&gt;</code>) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use <code>self</code>.
 
Example 1:
<pre>
player->AddItem, "Gold_001", 200
</pre>
is now:
<pre>
omw.additem("player","Gold_001",200)
</pre>
 
Example 2:
<pre>
AIWander, 0, 0, 0
</pre>
is now:
<pre>
omw.aiwander("self",0,0,0)
</pre>
 
Example 3:
<pre>
"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0</pre>
is now:
<pre>
omw.aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)
</pre>
 
===== Getting and Setting Variables =====
<pre>
omw.omwset("localvariablename", value)
omw.omwset("globalvariablename", value)
omw.omwset("objectID.variablename", value)
omw.owmset("globalID.variablename", value)
 
omw.omwget("localvariablename")
[etc.]
</pre>
Use <code>omwget</code> and <code>omwset</code> instead of <code>get</code> and <code>set</code> (for syntax consistency with OpenMW Python scripts; <code>get</code> and <code>set</code> are built-in functions in Python).
 
===== Non-extension Functions and Instructions =====
All extensions are directly available for use, but interpreter commands (like <code>MessageBox</code>) can be accessed with <code>omw.omwcall()</code>:
<pre>
omw.omwcall("MessageBox, \"This is a simple message\"")
</pre>
The argument to <code>omwcall</code> will be parsed as if it was a normal Morrowind script.


there's still a bunch of debugging output that will show up in the terminal, bear with it for the moment.
=== Debugging output ===
There's still a bunch of normal Python or Lua debugging output that will go to the terminal; bear with it for the time being.


Overview:
=== Security ===
* I think a post-1.0 goal, of mine at least, will still be to see if I can get an easy generic scripting interface.
Security will be an issue when it comes to running untrusted scripts, but it something people already deal with in the modding community (e.g, [https://docs.blender.org/manual/en/dev/advanced/scripting/security.html Blender Python Scripting Security]. Concerns can be addressed to an extent by making users opt in to using Python or Lua, and could be extended to only allow running scripts signed by a trusted certificate (key).
* It will be a seperate fork from openmw, as I think a unified scripting language and zini's enhancements to the existing implementation would be best for the community. It's generally better to improve one implementation rather than create several new ones. If, in the future, such extension of scripting is found desireable, then there is no problem incorporating it into openmw.
* There are some things that python/other languages can do better and can be implemented faster than writing a new scripting engine from scratch. For example, you can use all of python's existing modules and capabillities that could enable things like dynamic web content/interaction in games. Some people are already used to lua and python scripting, they could potentially write more complicated scripts or more easily integrate scripts. For discussion on python versus lua, I found this very helpful:http://lua-users.org/wiki/LuaVersusPython -- I think both could be usefull in different cases.
* Existing scripting IDEs can be used, most scripting language already has built-in verbose debugging. It's possible to generate embedded consoles for testing scripts on the fly. Scripts can be edited and recompiled on the fly, in game.
* Security will be an issue (running untrusted scripts), but it something people already deal with in the modding community (e.g, [http://wiki.blender.org/index.php/Doc:2.6/Manual/Extensions/Python/Security Blender Python Scripting)Security]. Concerns can be addressed at a minimum by making users opt in to using python, and can be extended to only allow running scripts signed by a trusted certificate/key.


Implementation:
== To do ==
https://github.com/maqifrnswa/openmw/tree/python-scripting
sandboxing Lua, options to enable/disable compile and runtime availabillity of scripts, code clean up, script signing/trust
* a new executable, extensionstool, generates bindings. A header and cpp file are generated containing that creates compiled code for every extension based on the arguments passed to the function, then sends the compiled code to an interpreter. The interpreter is declared by the console or by the new opcode, the bindings just have a pointer to the interpreter that is owned by whomever created it.
* SWIG compiles the bindings and creates a module _openmw.so and python module openmw.py.
* Right now, the console intercepts all *.py files so the console command "filename.py" will run the python script filename.py
* All commands are lowercase and take the same arguments as before. Commands that can take a reference (commands with ->) now have a first argument which is the reference. For example:
<pre> player->AddItem, "Gold_001", 200 </pre> is now <pre> additem("player","Gold_001",200)</pre>

Latest revision as of 19:41, 30 September 2018

Overview

These are notes and current status of extending openmw scripting through SWIG.

Current Status

Python and Lua scripting is fully functional. You can call from the console, or can attach to objects/global scripts with the command

StartExternalScript, "scriptname.py"

or

StartExternalScript, "scriptname.lua"

This way you get access to all the locals and globals as if your script is a normal script. The Python/Lua script has to be in the same directory that other morrowind data/addons are in (one of the data directories).

How to Install and Use

Install Prerequsites

Ubuntu Linux:
(Should also work in other Linuxes that use the apt package manager.)

$ sudo apt-get update
$ sudo apt-get install python3-dev swig liblua5.1-0-dev

Get the modified openmw binary

Github

Command-line example for Unix-like operating systems:

$ git clone https://github.com/maqifrnswa/openmw/tree/python-scripting 
$ cd openmw/
$ mkdir build && cd build
$ make -j

Ubuntu PPA

A Utopic personal package archive (PPA) is available for Ubuntu (and related) Linux:

$ sudo apt-add-repository ppa:showard314/openmw-scripting
$ sudo apt-get install openmw

It will install the highest version of openmw in all the repositories you have enabled, so if you have the openmw PPA, the daily PPA, and this PPA turned on, it may download versions from unwanted PPAs as each gets updated. Therefore, to be safe, only enable one PPA in the Software Center Sources at a time. This version also comes with "testing.py" as seen in the Example below.

Script Testing

Make your Python or Lua script; see Python example below. The test the script in the console. A command-line example for most Unix-like operating systems:

$ ./openmw --start="Seyda Neen" --skip-menu

Then in game, press ~ to get the Console, and run your script with the new command StartExternalScript, "testing.py" or StartExternalScript, "testing.lua", or write a script/addon that calls StartExternalScript.

Python Scripts

OpenMW will import your python script as a module and run the run() method when you use the Morrowind command StartExternalScript.

Example Script

Make sure you have a run() method, that is what will be called by OpenMW:

#Example file: "testing.py"
#SomethingScript is a script in a .omwaddon file that you create that has a short variable named "hello" you can use for testing.
#It's just a demonstration of setting a local variable, you can comment it out if it causes problems.
from openmw import *

def run():
    print("health: " + str((gethealth("player"))))
    print("hello i'm a criminal")
    setpccrimelevel(100000000)
    print("flying: " + str(getflying("player")))
    setflying("player",1)
    print("flying: " + str(getflying("player")))
    print("random100: " + str(omwget("random100")))
    omwset("random100",42)
    print("random100: " + str(omwget("random100")))
    print("SomethingScript.hello: " + str(omwget("SomethingScript.hello")))
    omwset("SomethingScript.hello",42.0)
    print("SomethingScript.hello: " + str(omwget("SomethingScript.hello")))
    omwcall("MessageBox, \"This is a simple message\"")

How to Write Python Scripts

All commands are lowercase and take the same arguments as before, even optional ones. Commands that can take a reference (commands with ->) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use self.

Example 1:

player->AddItem, "Gold_001", 200

is now:

additem("player","Gold_001",200)

Example 2:

AIWander, 0, 0, 0

is now:

aiwander("self",0,0,0)

Example 3:

"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0

is now:

aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)
Getting and setting variables
omwset("localvariablename", value)
omwset("globalvariablename", value)
omwset("objectID.variablename", value)
owmset("globalID.variablename", value)

omwget("localvariablename")
[etc.]

Use omwget and omwset instead of get and set to avoid name collisions with built-in functions in Python.

Non-extension Functions and Instructions

All extensions are directly available for use, but game interpreter commands (like MessageBox) can be accessed with omwcall():

omwcall("MessageBox, \"This is a simple message\"")

The argument to omwcall will be parsed as if it were a normal Morrowind script.

Lua Scripts

OpenMW will import your Lua script as a module and run the run() method when you use the Morrowind command StartExternalScript

Example Script

Make sure you have a run() method; that is what will be called by OpenMW:

#Example file: "testing.lua"
#SomethingScript is a script in a .omwaddon file that you create that has a short variable named "hello" you can use for testing.
#It's just a demonstration of setting a local variable; you can comment it out if it causes problems.

local testing = {}

function testing.run()
    print("health: "..omw.gethealth("player"))
    print("hello i'm a criminal")
    omw.setpccrimelevel(100000000)
    print("flying: "..omw.getflying("player"))
    omw.setflying("player",1)
    print("flying: "..omw.getflying("player"))
    print("random100: "..omw.omwget("random100"))
    omw.omwset("random100",42)
    print("random100: "..omw.omwget("random100"))
    print("SomethingScript.hello: "..omw.omwget("SomethingScript.hello"))
    omw.omwset("SomethingScript.hello",42.0)
    print("SomethingScript.hello: "..omw.omwget("SomethingScript.hello"))
    omw.omwcall("MessageBox, \"This is a simple message\"")
end

return testing

How to Write Lua Scripts

All commands are in the omw namespace, are lowercase, and take the same arguments as before, even optional ones. Commands that can take a reference (commands with ->) now have a first argument which is the reference. That first argument is required. If it is an implicit reference, use self.

Example 1:

player->AddItem, "Gold_001", 200

is now:

omw.additem("player","Gold_001",200)

Example 2:

AIWander, 0, 0, 0

is now:

omw.aiwander("self",0,0,0)

Example 3:

"urzul gra-agum"->AIWander, 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0

is now:

omw.aiwander("urzul gra-agum", 128, 0, 0, 60, 30, 10, 0, 0, 0, 0, 0, 0)
Getting and Setting Variables
omw.omwset("localvariablename", value)
omw.omwset("globalvariablename", value)
omw.omwset("objectID.variablename", value)
omw.owmset("globalID.variablename", value)

omw.omwget("localvariablename")
[etc.]

Use omwget and omwset instead of get and set (for syntax consistency with OpenMW Python scripts; get and set are built-in functions in Python).

Non-extension Functions and Instructions

All extensions are directly available for use, but interpreter commands (like MessageBox) can be accessed with omw.omwcall():

omw.omwcall("MessageBox, \"This is a simple message\"")

The argument to omwcall will be parsed as if it was a normal Morrowind script.

Debugging output

There's still a bunch of normal Python or Lua debugging output that will go to the terminal; bear with it for the time being.

Security

Security will be an issue when it comes to running untrusted scripts, but it something people already deal with in the modding community (e.g, Blender Python Scripting Security. Concerns can be addressed to an extent by making users opt in to using Python or Lua, and could be extended to only allow running scripts signed by a trusted certificate (key).

To do

sandboxing Lua, options to enable/disable compile and runtime availabillity of scripts, code clean up, script signing/trust