Hi folks,
in the past there have been various discussions on which IDE to use, what compiler to prefer and how to get X to work with Y to achieve Z. Moreover, there have been discussions on how it is crucial for the success of a project that it targets the entire audience of X-Plane. This lead to heated discussions on cross-platform development.
I've been asked by two forum members independently on how to setup an IDE for cross-platform development. Instead of getting into lengthy discussions in private that are of no use to other forum members I decided to elaborate a bit more and write a complete guide. So please, don't ask questions via pm, but discuss in public where others can profit from everyones knowledge. I firmly believe that the only dumb question is the unasked one.
So lets get started. At first glance, X-Plane plugin development seems to be rather easy: All you have to do is to write some C sourcecode using a plain-text editor, point your compiler to the SDK headers, point your linker to the libs, tell the linker to create a shared library, hit enter and you're done.
But alas, it's a bit more involved when it comes to real life: There are three different platforms to handle, multiple source files have to be organized, sometimes third-party libs are needed. Doing everything "by hand" becomes a daunting task, managing your builds on three platform gets complicated. So we need a general way of getting things done that works EVERYWHERE and - even more important - everywhere IN THE SAME WAY.
From 10 years of experience in programming, thereof 5 in C++ and 3 in X-Plane plugin development, I condensed the following guide.
The essential parts of your development environment are:
-The editor or IDE - the frontend where you hack your sourcecode and press a key to get the build done
-The compiler and linker - they do the hard work in the background to transform your human-readable sourcecode into machine-readable binaries
-The build system - organizing your input and putting the compiler and linker to work
-The version control system - helps with your workflow over time and is invaluable even if you are a one-man project
The key point to set up a development environment for cross-platform development is to use the same tools on every platform. If you use different tools to get the same task done, you will be in the constant state of wondering "wait, how the heck did I always do this on Linux?". Instead, if your build environment consists of the same tools everywhere, developing will feel the same everywhere, greatly speeding up your progress.
The following setup is what I'm going to describe in detail:
-Using Qt-Creator as IDE, because it is easy to use and lightweight compared to the Eclipse monster
-Using gcc/ld as your compiler and linker on all platforms, because it is mature and standards-compliant
-Using qmake as build system, because it is easy to use and requires less fuss on windows than autotools
-Using git as version control system, because it is distributed and extremely fast - and its integrated in Qt-Creator
1. Source Code Management / Version Control
I will get into this topic from the least obvious part of your development environment, the version control system.
Why the heck would a plugin developer as a one-man show need a version control system, you may ask.
Well, let me illustrate your workflow without using a DVCS (distributed version control system):
You start development on your desktop computer that runs windows. You hack a few hours, build the plugin, test it, and also send it to friend to test it. Then you copy the stuff to your Macbook to run a build on Mac. The build fails and you start browsing through mailing lists to fix the error. You change a few lines of code, until the build succeeds. Meanwhile you get a mail from your friend who complains that the plugin does X when it should do Y. So you sit down at your windows PC, alter another few lines of code there, rebuild, and send it to your friend again. While you again work on the Macbook (you of course want to copy the changed files from your desktop PC real soon!) you get a phone call from a work colleague who tells you that you can speed up your code by using the XYZ pattern, and you want to try it immediately. As the result looks promising, you copy all this on a USB-drive which you drag to work where there is a big Linux box, to get the Linux build done. After that, you go on holiday for a week, and come back to find a lot of hate-mail in your Inbox from Linux-users who complain your plugin does X when it is supposed to do Y.
Oh dear, you can by no means remember what changes you applied to the code on your Windows machine to fix this bug.
This is what happens if you not use a DVCS, you end up copying dozens of files from one computer to another, having half a dozen copies of your sourcecode in various states of inconsistency and all this becomes an unmanageable mess. You create new bugs by fixing old ones through introducing inconsistencies you cannot keep track of.
Alas, the solution is a DVCS: You organize your code in a so-called repository. Perhaps the most important thing about repositories is that they have a history. Every time you make a change to your code, it is memorized. So when you find a bug in a part of your software that you know worked 2 weeks ago, you simply ask your DVCS "show me all changes to this piece of code that where made during the last two weeks" and you can review the whole history of how the bug was introduced. Also good to know is that repositories can be easily "cloned" to other computers and by other people. Because modern DVCSs track file-content instead of files, it is easy to "merge" changes together if you applied changes to the same file on different computers. So you always have a consistent codebase across different OSs and computers and you can always review the history. Moreover, you lose the fear of breaking your code by implementation changes. If it doesn't work out as it should, you can always revert to a previous state and start over. Trust me, you cannot achieve this at the same level of comfort by simply copying files, renaming them *_old.
Also, a distributed version control system eliminates the single point of failure, because you have a consistent codebase across different hard drives and you can easily dump your repo on your backup disk from time to time.
In fact, people use DVCSs to do backups, and some even take it to the length of versioning their system (or at least their homefolder) as a whole, thus putting configuration files, documents, photos and whatever under version control. I do this, and it greatly helped me organizing my backup strategy.
Okay, so you have come to the conclusion that using a DVCS is really a good idea. Now, which to use? The old school of version control is a tool called subversion, but it is a centralized version control system, as opposed to a distributed version control system. That puts you again to the "single point of failure" problem, and setting up an svn server can be a daunting task.
Representatives of the new generation of distributed version control systems are tools like git, mercurial and bazaar. Git is the most mature of them, very fast, efficient and extremely scalable. People argue that mercurial is more intuitive to use, but this is only true if you are a die-hard subversion user, because mercurial commands are almost identical to subversion commands.
Installing git is easy:
On Linux, it is most likely already part of your distribution, or available as a distribution package.
On Windows, you have to install MsysGit: http://code.google.com/p/msysgit/
On Mac, you get git for OSX here: http://code.google.com/p/git-osx-installer/
For distributing over the network you also need an SSH client. Linux and Mac usually come with one preinstalled, for windows, puTTY does this job.
For Windows, you also may want to install an Explorer extension. Though this is not necessary, most Windows users are daunted by git's command line options and prefer to have the most frequently used commands available in Windows Explorer at one click. The tool of choice here is TortoiseGit.
When you have installed git, the next step is to create an empty repository. Then we will start by importing the X-Plane SDK into it.
I will show you both the command-line and the Explorer extension way.
First, we introduce us to git with the name that is used to define our commits. As soon as you start sharing your project with other people, you want to know who changed a piece of code, so you are sure to slam the right person in the face when a new bug surfaces:
Lets start by creating a folder for our work, named, lets say, "xplugins", and initializing it as a git repository:
Now we unpack the SDK here
And add it to our repository:
Now comes the important part: We have to "commit" what we have done in our repository. Whenever you completed a task that is worth creating a point in history of the repository, you "commit" what you have done since the last point. Commits are done with a descriptive message of the task you just completed:
When to do a commit? Commits should be fine-grained, but consistent.
That means, it is worthless if you do a commit once a day in the evening, saying in the commit message "work done on day X", changing two dozen files. When tracking down a bug, it will be difficult to isolate because the history of your code consists of a few large blobs that are difficult to review. On the other hand, renaming a function declaration, doing a commit, then renameing it in the implementation and doing another commit is also nonsense, because the commit in between the changes is inconsistent. So, do a commit whenever you completed one logical coherent task. As a rule of thumb, I do a commit whenever the task I'm working on is in a state that I can safely interrupt work to have a cup of tea.
For the purpose of showing you how cloning works, suppose you realize the autumn weather outside is so great you decide to do the rest of the work with your notebook on the balcony.
To clone the repository to your notebook via SSH, do the following on your notebook
Where you obviously replace philipp@192.168.43.242/home/philipp/... with the user, ip-adress and path of the repo of your desktop computer.
Now you can continue work in the repository on your Notebook!
Lets create a folder for our first plugin, "hello_world" and add it to the repo:
Oh, WTF, it starts raining! We rush inside, saving our Notebook from the rain, and decide to continue working on the desktop computer. What we have to do, is to "pull" what we changed on our repo on the Notebook to the desktop. So, on the desktop PC, we do:
Obviously, the IP-adress and path must be that from our notebook repository.
Lets see what happened:
Congratulations, that's all you need to know about git for now. You just managed the most important task: Having a history of what you change, and a consistent codebase everywhere you want to work.
Of course, git can do far, far more than that. To learn what git can do for you, I suggest you read the git-tutorial and everyday git with 20 commands.
So congratulations if you made it until here! We didn't compile a single ###### plugin yet. Time to move to the next important part!
2. Breaking up Compiler, Linker and Buildtool
What most people refer to as "compiling" are actually two steps. To get a working plugin (or any working binary) out of C or C++ code, two things are done: First, the compiler creates an object file out of every compilation unit that is part of your plugin. Then those object files are fed to the linker, along with static libraries, which are just bundled object files. The linker stuffs all this together in a single binary file, that is your plugin.
Think of the compiler-linker toolset as your translator from human-readable code to computer gibberish. The problem with compilers is, that every one of them has a slightly different opinion on how to interpret what you write. That means, code that works with one compiler might not work with another. Therefore it is crucial to have a compiler that is available on every platform, so you code once and compile everywhere (in theory). It is also advisable to have more than one compiler installed and regularly compile your sourcecode on either compiler. That is a good practice to make sure your code is as standards-compliant as it can possibly be, and that you don't rely on a certain compilers (perhaps faulty) behavior.
To achieve this, we will later decouple the build process from the actual compiler used.
But first, what is the compiler of choice, that is available on every X-Plane platform? Obviously, this is gcc, the GNU compiler collection. Mature, outstanding standards compliance, available on all platforms supported by X-Plane. As a second compiler for crosschecking our code we will use MS Visual C++. Question to the more advanced readers: Did anyone try to build an X-Plane plugin with icc, Comeau or LLVM/clang yet?
So the next step in setting up our environment is to install gcc, or more precisely, the GNU toolchain, which consists of gcc, binutils, make, ...
On Linux, all this stuff is almost certainly already installed, or you can pull it in by one click in your packet manager.
On Mac, just install the Developer Tools from your OSX DVD, they contain gcc.
On Windows, there are various distributions of the gcc bundled in "Minimalist GNU for Windows" or short "MinGW". The official minGW distribution is a terrible mess of a lot of files without an automated installer and you have no idea which of the 54173 different files you have to download. Instead, you should download the Qt-SDK for Windows. It contains various components some of those I will discuss later, but among them is a bundled, state-of-the-art gcc and it comes with a foolproof installer. You can get it here: http://qt.nokia.com/downloads/sdk-windows-cpp
While we are at it, we also want to install MSys (minimal system) for windows, so that we have a more powerful shell compared to cmd, and also to have the same work environment as on Linux and Mac. Remember what I said about having the same tools for the same job. So download msys, and install it. During installation you will be asked where you installed minGW. If you used the default that the Qt-SDK suggested, the location will be C:\Qt\2010.05\mingw\
Still on windows, we also want to install Visual C++ as our second reference compiler.
Many people on Windows refer to Visual Studio or Visual C++ as their "compiler". More correctly, we have to refer to Visual C++ as their IDE, because Visual C++ is an enormous bunch of different tools working together. What we care for here, are just two tiny tools of the Visual C++ suite, cl and link. We won't be using the IDE at all, just the command line tools cl, which resembles g++ from gcc and link, which resembles ld from gcc.
Okay, now we have our set of compiler and linker on every platform, and our second reference compiler on windows, how do we get them to actually compile our sourcecode?
3. The Buildtool(s)
Let me illustrate this at the lowest level: Getting the g++ compiler to compile the HelloWorld of the SDKExamples requires just two simple, easy-to-remember commands on the terminal. So in your shell you do:
Easy, to remeber, isn't it? Of course, the commands are slightly different for a Mac, slightly different on Windows with gcc, and completely different on Windows with VC++. Seriously, WTF???
The first step of automating the build somewhat is using a tool called "make" by gcc (and "jom" by Visual C++) to invoke the tools in the right order on the different files your plugin consists of. Make is a kind of scripting tool that reads "Makefiles" where you define what you'd like to get done and then puts the compiler and linker to work. So the task that remains is to create those Makefiles. Unfortunately, these Makefiles will again be different for Windows, Linux and Mac. If we'd code them by hand, it is more than likely that when we apply a quick change to one of them, we will forget to update the others.
So the ultimate step in automation is using a tool that will generate the Makefiles for us. There are numerous approaches to that, autotools ("configure&&make") being the most prominent on Mac and Linux. But using autotools on Windows is a daunting task and requires you to install a metric ton of other tools, like cygwin. Not exactly what we want.
But we already have everything installed that we wanted! Remeber the large "Qt SDK" we installed on Windows when we wanted to have gcc? Now it is time to look what other things are part of the Qt SDK. The most important for us is the tool called "qmake". It is an incredibly simple whilst powerful tool. You describe what you want to have build in a "project file" (.pro), and then qmake generates the right Makefile for your platform.
So instead of remembering four different sets of commands, or managing four different Makefiles, we just define one "project" and let qmake generate the necessary Makefiles for us. When qmake has generated the Makefile, we just say "make" and the build process is done for us. In the end, we will just hit Ctrl+B in our IDE and qmake, make, gcc and ld will be put to work without requiring us to remember all the low level stuff.
Here is what a project file for an X-Plane plugin looks like:
This is the template for every plugin project that we want to start. Everything peculiar to the X-SDK, like the platform-defines and the special options for Mac, is in it. And it's the same for every platform. Whether you are on Linux, Windows or Mac, you tell qmake to use this file, and you will get a working plugin. It will also take care of building a universal (PPC+Intel) plugin on Mac.
And moreover, this file doesn't care for whether we are using gcc or Visual C++. qmake will generate the necessary Makefile for either tool out of this file, and allow us to compile the source code on either compiler without further ado.
On Windows, we already have the Qt-SDK installed along with MinGW. For using it with Visual C++ also, we don't need to download the whole SDK again, just the part compatible with Visual C++. To do so, download and install: http://qt.nokia.com/downloads/windows-cpp-vs2008
On Mac, we install the SDK http://qt.nokia.com/downloads/sdk-mac-os-cpp
On Linux, if qmake is not already installed (which it is most likely on a KDE distribution, less likely when using Gnome) you need to get the packages named libqt4-devel or qt4-dev or something the like, varying for your distribution.
So, lets try it out. Copy the contents of the HelloWorld.c from the SDK Examples to your empty hello.cpp, save the above project description as "hello.pro", and change the line SOURCES to just include SOURCES += hello.cpp.
Ready to go? Just start qmake in the directory where the .pro lies, and everything else is done for you!
There it is! Qmake generated the Makefile with all the commands to get compiler and linker to work. Now we tell make to just do that!
This "liblin.xpl.so" is your plugin! Rename it to "lin.xpl", shove it into a subfolder of X-Plane/Resources/plugins and there you go! (On Mac, it will be named libmac.xpl.dylib, on Windows win.xpl1.dll, you raname to mac.xpl and win.xpl, obviously).
So, are you ready for the ultimate step, setting up the Qt-Creator IDE to do all the work for you?
4. Setting up the IDE
So, fire up Qt-Creator and import our project file.
You will then be asked to import the existing build settings. Click yes.
Now we have imported the setup that qmake just created. Now we can go on coding on our plugin. A build is now easy: Press Control+B or click the small hammer on the left bottom. The output of the compiler will then be displayed at the bottom, just like it was before on the console.
You can do all the git tasks from within Qt-Creator, like committing, reading the history or reverting to a previous version.
Ready for the ultimate overkill? Lets create a shadow build!
Remember, on windows we have two toolchains:
the mingw: qmake->make->g++->ld
and the vc++: qmake->jom->cl->link
The Mingw toolchain is already known to Qt-Creator, because it was part of the SDK download. Now we enable the VC++ toolchain for Qt-Creator:
Click Tools->Options and go to the "Qt4" tab.
Click the plus button and navigate to where you installed the Qt-Libraries for MSVS (the second download from the Qt website). Go to the bin folder, select qmake.exe and click open.
You might want to give it a descriptive name with "msvc" in it, so you can distinguish it from your other Qt installation.
Then hit OK.
Now it easy to compile with either of the two toolchains:
Click the "Projects" tab on the left side of Qt-Creator.
At the "Qt version" dropdown list you can now select either of your installed Qt-toolchains.
Now click "Build"->"Rebuild all" to commence a build with Visual C++.
Compiling with multiple toolsets has never been so easy.
The crucial point here, is that you don't have to switch the IDE. You always work in the same environment, you don't have to learn to use different IDEs - Qt-creator is everywhere you work, and even if you want to use the Visual C++ toolchain, you can do it out of Qt-Creator. This is what I call comfort!
Lets see what else you can do with Qt-Creator. First, lets automate the dumb process of having to rename the target and copy it to X-Plane. We do this by going to the "Projects" tab and selecting "Add Build Step"->"Custom Process step". Check "enable" and add a mv command to move liblin.xpl out of the $BUILDDIR to the $XPLANEDIR/Resources/plugins/$PLUGINNAME
Then scroll down to "Build environment", click "Details" and add a variable XPLANEDIR and store the path to your X-Plane installation in it.
Add another Variable PLUGINNAME, that will be the sub directory of Resources/plugins in which your fat plugin will reside.
Okay, now your plugin gets moved to the appropriate folder and is renamed to the name that X-Plane expects (Obviously, the renaming on Windows an Mac is different, where the plugin must be called win.xpl and mac.xpl accordingly!).
Now select the "Run settings" tab. Here we declare X-Plane as the application to be started. So select your X-Plane-i686 or X-Plane.exe or X-Plane.app/Contents/MacOS/X-Plane as the application to start and choose the X-Plane dir as work dir. I also like ot select "run in terminal" to see all the debug output printed to the console.
So now the build process is completely automated: When you hit Ctrl+R (or click Build->Run), your plugin gets compiled, moved to X-Plane, and X-Plane is started afterwards.
The last step missing now is to setup the debugger. But this is easy. So when X-Plane was started (by Ctrl+R), click Debug->Start Debugging->Attach to Running External Application and select X-Plane from the lists of processes. Voila, you can now setup breakpoints in your plugin and start debugging with the marvellous graphical GDB frontend of Qt-Creator.
Congratulations, you are done. You have now a cross-platform development setup with a full-fledged IDE, debugger, two different toolchains and a source code management system.
Philipp