Development Environment Setup

From OpenMW Wiki
Revision as of 16:44, 9 November 2018 by AnyOldName3 (talk | contribs) (What you do)

Jump to: navigation, search

This page will describe how to set a dev env for several OSes.

Third-Party Libraries and Tools

  • OpenMW additionally uses the following tools:
    • CMake - used to manage the build process
    • Git - used as the source control management system

For those unfamiliar with Git, the Sphere RPG Engine group has a fairly useful short wiki page entitled "Git for the lazy". Make sure to also check out the GitLab FAQ.

General notes on dependencies

MyGUI

By default, MyGUI will build with the Ogre3D platform enabled, but Ogre3D is not a hard dependency. If you do not want to have Ogre installed, then configure MyGUI with the dummy platform, and disable tools & demos:

cmake .. -DMYGUI_RENDERSYSTEM=1 -DMYGUI_BUILD_DEMOS=OFF -DMYGUI_BUILD_TOOLS=OFF -DMYGUI_BUILD_PLUGINS=OFF

OpenSceneGraph

It is recommended to use the OpenMW/osg fork of OpenSceneGraph. OpenMW still works with "vanilla" OpenSceneGraph, but may run at slightly lower frame rate.

OSG plugins

OSG comes with numerous plugins, most of which are not required by OpenMW. If you install OSG from a package manager, then you will have tons of bloat on your system that OpenMW does not need (another reason to build your own).

Only the following plugins are required for OpenMW to function properly:

Texture formats:

  • osgdb_dds
  • osgdb_tga
  • osgdb_bmp
  • osgdb_jpeg
  • osgdb_png

.osg format support:

  • osgdb_osg.so
  • osgdb_serializers_osg.so

Note, the jpeg/png plugins will not be built unless you have 'libpng' and 'libjpeg' headers installed on your system.

If you are creating a release package, the library files for the plugins need to go in <library path>/osgPlugins-<OSG_VERSION>.

When using the OSG git master (to be 3.6), or the OpenMW OSG fork, you can use the following cmake arguments to avoid unneeded plugins from being built:

-DBUILD_OSG_PLUGINS_BY_DEFAULT=0 -DBUILD_OSG_PLUGIN_OSG=1 -DBUILD_OSG_PLUGIN_DDS=1 -DBUILD_OSG_PLUGIN_TGA=1 -DBUILD_OSG_PLUGIN_BMP=1 -DBUILD_OSG_PLUGIN_JPEG=1 -DBUILD_OSG_PLUGIN_PNG=1 -DBUILD_OSG_DEPRECATED_SERIALIZERS=0

S3TC texture compression support

OpenMW requires S3TC texture compression support. If you are using a proprietary video driver, you will usually have S3TC support out of the box. With open-source drivers on Linux, you may need to install S3TC support separately. An indicator for missing S3TC support are completely white or pink textures.

On Ubuntu:

sudo apt-get install libtxc-dxtn-s2tc0

Windows

Visual Studio 2013/2015/2017 scripted way

This is the recommended method to build OpenMW on Windows. This script is kept updated and has fewer ways of going wrong than setting up your development environment manually. If you have issues, make sure to report them somewhere official, such as the OpenMW forums, as otherwise we won't know anything needs fixing.

What you need

  • Visual Studio 2013, 2015 or 2017 (Community edition is enough)
    • Make sure you've got the C++ toolset installed, which isn't necessarily installed by default.
      • In VS 2017, this is called "Desktop development with C++" and is available under the workloads tab in the Visual Studio installer. This can be added during initial installation or alternatively afterwards by typing "Visual Studio Installer" into the start menu search and choosing to modify Visual Studio 2017.
      • In VS 2015, a similarly-named package is available and can be added during initial installation or afterwards by choosing to repair then modify your VS installation from the Windows Programs and Features menu.
      • VS 2013 should always come with C++ support.
  • MSYSGit or Git for Windows as it's also called (cinst git.install if you have Chocolatey)
    • Other Git implementations will be sufficient to clone the OpenMW repo, but MSYSGit also provides the bash shell used to run the script.
      • Note that WSL does not work for this as the script needs to execute windows executables.
      • Note that a full Cygwin install does not work for this as it tries to emulate Unix file permissions, and the script was not written with this in mind.
  • 7-zip in your path
    • cinst 7zip.install works if you have Chocolatey, but make sure you reload your shell or otherwise refresh your environment variables afterwards.
    • If you don't use Chocolatey, install 7zip like any other application, then use the instructions in this guide to add it to your path.
  • CMake in your path
    • cinst cmake.install --installargs 'ADD_CMAKE_TO_PATH=User' for Chocolatey, but make sure you reload your shell or otherwise refresh your environment variables afterwards.
    • If you don't use Chocolatey, install CMake like any other application, making sure to choose "Add CMake to the system PATH for all users" or "Add CMake to the system PATH for the current user" when offered.
  • ~10GB of free space per build platform (32/64-bit), for all the build artefacts.
  • Time. Around an hour most likely for a full release or debug build.

What you do

  • Pick a directory to do this in.
    • Its path cannot contain spaces (this can't change until Qt can be installed with a path containing spaces).
    • Its path should either be short (e.g. C:\OpenMW-dev\) or you should enable long file paths in your registry (for example by following this guide).
  • Clone the OpenMW repo. git clone https://gitlab.com/OpenMW/openmw/ (or your own fork of it).
  • Open a Git Bash shell in the repo.
    • Remember that other Bash implementations such as WSL or Cygwin won't work.
  • Run the following command: CI/before_script.msvc.sh -k -p Win64 -v 2017.
    • Substitute Win64 with Win32 if doing a 32-bit build.
    • 2017 can also be substituted with 2013 for VS2013 or 2015 for VS2015
    • This will only install the runtime resources for the Debug configuration, so while others will be available in Visual Studio, they won't run.
      • To install them for Release instead, add -c Release to the command.
      • To install them for RelWithDebInfo instead, add -c RelWithDebInfo to the command.
      • If we ever change the default, adding -c Debug will set up the Debug configuration.
      • Only one configuration can be set up per command invocation, so feel free to run it again with another configuration specified to get all the ones you need.
  • Wait for the build environment to be set up.
  • Go to the build folder that was just created (MSVC2017_64 if you've used VS 2017 and specified 64-bit) and open OpenMW.sln.
  • If you're using VS 2017 and this Visual Studio bug hasn't been fixed yet, there's an extra step to work around it.
    • In Visual Studio, for each executable target you intend to use (e.g. openmw, openmw-cs, openmw-launcher etc.), right click on it in the Solution Explorer and choose Properties to open its Property Pages.
    • At the top left of the popup, you'll see that Configuration is set to something. Changing this to All Configurations ensures that you won't have to repeat this for every single build type later.
    • In the left hand list, under Configuration Properties, there'll be a Debugging category. Select it.
    • In the right hand list, you'll see that Working Directory is set to $(TargetDir). Change this to something else, then press Apply at the bottom right of the popup.
    • Change it back to $(TargetDir) and then press Apply again.
  • Make sure the Configuration dropdown at the top of Visual Studio is set to the one you want to use.
    • Debug builds are slow, but the easiest to use with a debugger.
    • Release builds are fast, but cannot be effectively used with a debugger.
    • RelWithDebInfo builds are fast, and can be used with a debugger, but some lines may be executed in a different order to how they appear in the source code, and some may be skipped altogether, making debugging more difficult.
  • If you want to use Visual Studio's debugger, make sure you've set a startup project in the Solution Explorer by right clicking one and choosing Set as Startup Project. (The default startup project is ALL_BUILD which doesn't have an executable as it's a meta project used to build all other OpenMW projects at once, so cannot be started.)
  • Develop.

If this doesn't work

First, double check that you've followed the instructions properly. Some common problems are in bold. If you still have issues, make sure to report them somewhere official, such as the OpenMW forums, as otherwise we won't know anything needs fixing.

Updating dependencies

Once in a while, the versions of dependencies listed in the script will be updated. For some of these, they can be updated just by running the script again. However, for some others, the process has not been automated yet. The rough steps to do so manually are as follows:

  • If you don't have any particular reason to keep the existing build directory (the MSVC<year><_64> directory), simply delete it and use the script to regenerate it. You will get a new build environment using updated dependencies.
  • If you need to keep the build directory, the process is more involved:
    • Delete the directory the dependency was extracted to (MSVC<year><_64>/deps/<dependency name>)
    • Remove any lines mentioning the dependency from MSVC<year><_64>/CMakeCache.txt
    • Run the script again.


Previous versions of this guide

https://forum.openmw.org/viewtopic.php?f=6&t=4074 quick placeholder

https://gist.github.com/ananace/de7f30454e99b3952405

Visual Studio 2008/2010/2012/2013

Setting up a Windows development environment manually is not recommended. If one were to follow these instructions, it would not lead to a working OpenMW build. If you have experience setting up the required libraries (including the ones not listed here), you may be able to work around any issues, but it's also possible that you won't be able to. It is unlikely that trying is worth your time.

Tools

  • Visual Studio 2008/2010/2012/2013 - normal or free (C++) express versions. Note that if you use Visual Studio 2012 or newer you will have to build Qt yourself.
  • CMake: CMake at least v2.8, but grab the latest stable release.
  • Git - Our version control software: Git at least v1.8.x, but grab the latest stable release.

Libraries

(Needs updating, +osg)

  • Boost: Use the installer for your Visual Studio version and set the environment variable BOOST_ROOT to the <install_path>.
  • Bullet Download bullet and build the BulletCollision, BulletDynamics and LinearMath projects. Make sure the Runtime library is set to "Multi-threaded Debug DLL (/MDd)" or "Multi-threaded DLL (/MD)" for the debug and the release configuration, respectively. Also "Enable Run-Time Type Information" has to be set to "Yes (/GR)". Afterwards put the libraries into the <install_path\lib\Release> folder, strip the ending off (like _vs2010) and set the environment variable BULLET_ROOT to the <install_path>.
  • Freetype: Freetype2, and set the environment variable FREETYPE_DIR to the <install_path>.
  • MyGUI: Download and build following compilation instructions and set the MYGUI_HOME environment variable.
  • OpenAL: Download OpenAL11CoreSDK and set the environment variable OPENALDIR.
  • Qt. Download the latest Qt4 version for your Visual Studio version and install the environment variable QTDIR. Example: qt-win-opensource-4.8.4-vs2010.exe
  • msinttypes Download and extract to your visual studio 'install_root\VC\include' directory and overwrite the files there. This will give you C99 compliant inttypes. This is only required for building FFmpeg and not required for Visual Studio 2012 or later.
  • FFmpeg Download and extract the dev and shared of ffmpeg. The latest builds don't work. A known working version is commit 1eabd71, available in the "previous versions" links on that page. The environment variable is FFMPEG_HOME. (Note: A later build ffmpeg-20120914-git-8bdba0b-win64 appears to work ok with openmw-0.30 without crashing)
  • SDL2 Download and extract the SDL2 VS "Development Libraries". SDL2-devel-2.0.1-VC.zip Then set the SDL2_LIBRARY variable to the location of the SDL2.dll file.


Once this is done, you're ready to get the source code.

Download and install all of these and make sure the environment (the system and Cmake) knows where they are. If you develop under a non-admin account, use System variables rather than User variables (in Control Panel->System and Security->System->System Properties->Advanced (tab)->Environment Variables). Add the following, adjusted to the paths in your system:

BOOST_ROOT  D:\sdk\boost-1.55
BULLET_ROOT D:\sdk\bullet
FFMPEG_HOME D:\sdk\ffmpeg
MYGUI_HOME  D:\sdk\MyGUI_3.2.0
OPENALDIR   D:\sdk\OpenAL
QTDIR       D:\build\qt-4.8
SDL2DIR     D:\sdk\SDL2

Check that PATH includes the path to git.exe for CMakeGUI to find it.

Bullet libraries won't be found by CMake if you didn't put the built libraries manually into <BULLET_ROOT>\lib\Release\ as indicated above. Alternatively modify cmake\FindBullet.cmake to look for ${BULLET_ROOT}/lib.

Similarly, either set MYGUI_INCLUDE_DIRS and MYGUI_PLATFORM_INCLUDE_DIRS manually in CMakeGUI or modify cmake\FindMyGUI.cmake to look for ${MYGUISDK}/include/MYGUI.

Need to remove SDL2main.lib from VS->Properties->Configuration Properties->Linker->Additional Dependencies if you run into linking errors building owmlauncher. One way is to edit CMakeLists.txt so that SDL2_BUILDING_LIBRARY is set before running CMakeGUI. Another way is to set the SDL2_LIBRARY env variable instead of SDL2DIR as indicated above.

set(SDL2_BUILDING_LIBRARY ON)
find_package(SDL2 REQUIRED)

Running and using CMake

First, before opening Cmake, create a sibling directory called "OpenMW - build" next to the root "openmw" so that an out-of-source build can be done. This step is not needed if you're using CMakeGUI, as a separate build directory can be specified at the start.

 "<Cmake's Install Directory>\bin\cmake.exe" ..\openmw -G "Visual Studio 9 2008"

This will create a Visual Studio solution which you can use to compile OpenMW. (Hint: If you are building a debug version, add /bigobj to openmw project in VS->Properties->Configuration Properties->C/C++->Command Line->Additional Options)

Git on Windows

via Cygwin

Git is available on Windows via cygwin. Git can be used directly on Windows instead of grabbing source zip files from Gihub. The cygwin environment will allow you to use the same git command line commands as are recommended for the Linux environment. Note however that the cygwin version of CMake does not appear to support VS2008 as a valid target; therefore, cygwin can only be used for Git commands and a standard Windows command prompt should be used for invoking CMake.

via msysGit

There is also a stand-alone compiled package called msysgit. This will give you a Cygwin-like console with the git command, which you may find easier than via Cygwin.


via TortoiseGit

There is also a GUI git client for the one used to TortoiseSVN or TortoiseCVS which is called TortoiseGIT. It will give you a nice GUI inside the Windows explorer. This is the easy way.

via Eclipse

First of all, you need the egit plugin. Install it, restart eclipse, "file -> import -> git -> clone"

Linux

Installing Dependencies

Arch

There is a development package available for OpenMW on the AUR. It can be found at [1].

Information about building packages from the Arch User Repository (AUR) can be found at the Arch wiki.

Ubuntu

Key requirements

Because some dependencies of OpenMW are not in the Ubuntu repositories we made a development repository on Launchpad which provides packages for the missing dependencies.

To be able to install these packages you need to add the openmw Launchpad PPA by running:

sudo add-apt-repository ppa:openmw/openmw

and to make sure that the ppa will be used run:

sudo apt-get update

Packages

You will need g++ in order to compile OpenMW.

To setup a build environment and install all dependencies type:

 sudo apt-get install git libopenal-dev libopenscenegraph-3.4-dev \
 libsdl2-dev libqt4-dev libboost-filesystem-dev libboost-thread-dev \
 libboost-program-options-dev libboost-system-dev \
 libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev \
 libbullet-dev libmygui-dev libunshield-dev cmake build-essential \
 libqt4-opengl-dev

Debian Stretch

Debian Stretch has all the packages necessary to build OpenMW.

Install OpenMW dependencies

sudo apt-get install git libopenal-dev libopenscenegraph-3.4-dev \
libsdl2-dev libqt4-dev libboost-filesystem-dev libboost-thread-dev \
libboost-program-options-dev libboost-system-dev \
libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev \
libbullet-dev libmygui-dev libunshield-dev cmake build-essential \
libqt4-opengl-dev


Fedora Workstation

Key requirements

Because some dependencies of OpenMW are not in the Fedora Workstation repositories, you need to enable RPMFusion free and non-free repositories which provide packages for the missing dependencies.

Packages

You will need g++ in order to compile OpenMW.

To setup a build environment and install all dependencies type:

sudo dnf groupinstall development-tools; 
sudo dnf install openal-devel OpenSceneGraph-qt-devel SDL2-devel qt4-devel \
boost-filesystem git boost-thread boost-program-options boost-system ffmpeg-devel \
ffmpeg-libs bullet-devel gcc-c++ mygui-devel unshield-devel tinyxml-devel cmake

Fetching the source

Download the source from the main downloads page: [2], or get the latest git snapshot:

 git clone https://gitlab.com/OpenMW/openmw

 cd openmw

Building

With make

Note that this is an out-of-source build. Generally, this is the preferred method when working with cmake.

 mkdir build
 cd build
 cmake ../

This creates a regular makefile.

Now we can start compiling. Normally you can just run

make

but if you have multiple cores, you can decrease your compile time by doing this:

make -j`getconf _NPROCESSORS_ONLN`

macOS

Prerequisites

You must have recent version (7.0 or higher) of Xcode & Xcode command line tools.

Miminal macOS version is determined by minimal Xcode version. Supported deployment target is 10.8.

CMake & pkg-config are required. You can install these via Homebrew:

$ brew install cmake
$ brew install pkg-config

Dependencies

The easiest way to handle OpenMW dependencies is to use OpenMW-deps-Mac, which is what is used for official release builds. Either follow the instructions in that repository to build from source, or download the prebuilt dependencies here.

Qt

Supported Qt version for macOS builds is 5.5.1. Newer versions require C++11 to be enabled.

Note that you can install Qt from homebrew too but we previously had some issues with deployment of OpenMW binaries built against homebrewed Qt. It's completely fine if you don't intend to deploy to other machines, though.

$ brew install [email protected]

It's recommended, however, to use package from Qt Project site if you're planning to deploy your builds to other machines.

Build

First of all you'll need to clone OpenMW source code, either from our main repository or from your fork:

$ git clone https://gitlab.com/OpenMW/openmw/

Then you can cd into source directory and look around. Please note that out-of-source build is recommended:

$ mkdir build
$ cd build

I assume that openmw & build directories are siblings. Call CMake:

$ cmake \
    -D CMAKE_PREFIX_PATH="$PATH_TO_DEPENDENCIES_DIR;$PATH_TO_QT_INSTALLATION" \
    -D CMAKE_OSX_SYSROOT="$MAC_OS_VERSION" \
    -D DESIRED_QT_VERSION=5 \
    -D CMAKE_BUILD_TYPE=Debug \
    -D BUILD_MYGUI_PLUGIN=FALSE \
    -G"Unix Makefiles" \
    ../openmw/

Where $PATH_TO_DEPENDENCIES_DIR and $PATH_TO_QT_INSTALLATION are directories listed earlier in this guide. $PATH_TO_DEPENDENCIES_DIR is a directory like "openmw-deps" containing "lib" and "include" sub-directories, and $PATH_TO_QT_INSTALLATION is a path to a Qt 5.5.1 installation (for example ~/Qt/Qt5.5.1/5.5/clang_64/lib/cmake). Note that these paths must be absolute; if they are relative paths then the correct files will not be found.

$MAC_OS_VERSION is the version of macOS in the format macosx10.##. For instance, if you are running macOS 10.13 (High Sierra), $MAC_OS_VERSION would need to be set to macosx10.13.

You may need to include this line in the `cmake` arguments: -D SDL2_INCLUDE_DIR="$PATH_TO_DEPENDENCIES_DIR/include/SDL2" \.

You may need to brew install bullet.

Then build:

$ make

And try:

$ open OpenMW.app
$ open OpenMW-CS.app

Or run binaries directly to see the output:

$ OpenMW.app/Contents/MacOS/openmw-launcher
$ OpenMW.app/Contents/MacOS/openmw
$ OpenMW-CS.app/Contents/MacOS/OpenMW-CS

Deployment

If you want to make a release build and deploy it, you must set CMake variable OPENMW_OSX_DEPLOYMENT to TRUE:

$ cmake \
    -D CMAKE_PREFIX_PATH="$PATH_TO_DEPENDENCIES_DIR;$PATH_TO_QT_INSTALLATION" \
    -D CMAKE_OSX_SYSROOT="$MAC_OS_VERSION" \
    -D CMAKE_OSX_DEPLOYMENT_TARGET="10.9" \
    -D DESIRED_QT_VERSION=5 \
    -D CMAKE_BUILD_TYPE=Release \
    -D BUILD_ESMTOOL=FALSE \
    -D BUILD_MYGUI_PLUGIN=FALSE \
    -D OPENMW_OSX_DEPLOYMENT=TRUE \
    -G"Unix Makefiles" ../openmw/

If these instructions aren't up to date, refer to the CI scripts in the repo.

Build & package:

$ make package

After this command completes, you'll have a *.dmg file in current directory. You can publish this file as OpenMW release, send it to your friend, etc.

Note that if you'll try to launch release versions of OpenMW.app or OpenMW-CS.app directrly from build directory they won't work, because they're built in deployment configuration when all dependencies are expected to be inside application bundle, but they're not until everything is packed in *.dmg.

Troubleshooting

If you experiencing configuration or build errors, or just in doubt, please refer to our Travis CI build script, it's proven to be working and should help you setup build environment on almost any compatible Mac with OS X 10.9 or higher (older versions could work too, but we're unable to test on them).

Umbrella script: https://gitlab.com/OpenMW/openmw/blob/master/.travis.yml

Dependency setup: https://gitlab.com/OpenMW/openmw/blob/master/CI/before_install.osx.sh

Build: https://gitlab.com/OpenMW/openmw/blob/master/CI/before_script.osx.sh

Data

For a detailed description of how to obtain the data files check Installation_Instructions#Install_Game_Files.