Building PostgreSQL on UNIX-like systems from source code can be hard the first time. But often, once you extract the code it only takes three steps common to many C programs:
./configure
make
make install
With PostgreSQL 9.3.0 officially released, I’ve been testing out the final source code on some systems it was still a bit too raw for before now. And I ran into two problems: one real that I found a simple fix for, the other self-induced (but still painful).
This was on a FreeBSD 8.2 based system. The main lesson here is pretty simple: when building PostgreSQL on FreeBSD, always prefer gmake to regular make. There is sometimes a BSD make compatible file in a directory, but that’s not the case for all of the contrib module source code.
Checkout vs. release source code files
When you build a checkout of PostgreSQL from source code, it expects a few more development tools than the officially packaged versions. A random git checkout needs programs like bison and flex installed. Those are used to rebuild the database’s SQL parser. One step of the release build process does that for you and publishes the results. That means a release source code archive, sometimes called a “tarball”, requires less development tools.
Partly due to this issue (which I’ll return to again at the end), when new versions of PostgreSQL come out–like the just released version 9.3–I will test them on systems that don’t have a full development tool set. You still need a lot of things to build the database from source: a C compiler, perl, the make program, auto-configuration tools, and some other things. On this FreeBSD system I installed all of those one at a time until I had a minimal working set, and the main database code built and installed fine.
contrib modules
Once the main database is installed, there are also a set of additional programs called the contrib modules that you get source code for. The idea is that these programs aren’t required for the core database, but they can add extra features you might want. Today I needed the pgbench contrib program. Normally you just go into its directory do the usual make/make install combo. But not today:
$ cd contrib/pgbench
$ ls
Makefile pgbench.c
$ make
"Makefile", line 12: Need an operator
"Makefile", line 15: Could not find
"Makefile", line 16: Need an operator
"../../src/Makefile.global", line 45: Missing dependency operator
"../../src/Makefile.global", line 48: Need an operator
make is one of the lowest level C programming utilities. It lets you compile individual programs and specify how they fit together. If you use a server’s default make program, you can name the make script “Makefile” or “makefile” instead, as done here. To understand what’s gone wrong with it, you need some UNIX history.
make versions
FreeBSD traces its style back to the older BSD UNIX distribution. Some of the tools it provides are still based on expectations set by BSD UNIX. FreeBSD’s make to build software is one tool like that. It has its own specific format too–using the name “BSDmakefile” says you’re expecting the feature set BSD make supports. But here it’s just the generic makefile it’s choking on something, when being read by BSD make.
When you’re on a Linux system, you don’t get a BSD make utility. Instead you get the GNU Make utility. That has some common heritage with BSD make, but the two programs accept different command line parameters and syntax inside the build instructions. There are a lot of utility pairs like this. As another example, FreeBSD has its own tar utility for making file archives. There is also a GNU tar utility, which some prefer to the BSD one, but you won’t find it on most FreeBSD system.
GNU Make is also installed on most FreeBSD systems, because it’s a building block for so many popular GNU packages. But on FreeBSD the program is called gmake to tell it apart from the default, regular BSD make. Because the programs are a little different, if you want something specific to GNU make it looks for an input file named “GNUmakefile”.
But PostgreSQL needs GNU Make to build correctly. In the root of the source code tree, it has a Makefile for compatibility with other system make programs, but it’s just a wrapper. Its comments read like this:
$ more Makefile
# The PostgreSQL make files exploit features of GNU make that other
# makes do not have. Because it is a common mistake for users to try
# to build Postgres with a different make, we have this make file
# that, as a service, will look for a GNU make and invoke it, or show
# an error message if none could be found.
If you look inside of contrib/pgbench, there is a Makefile there. That means the BSD make program will try to read it. But the build instructions in there are actually written for GNU make. That’s why BSD make is choking on them.
Compiling with GNU make
It’s taken a long explanation to get to here, but the fix is simple. Just run gmake directly instead:
$ cd contrib/pgbench
$ gmake
$ gmake install
gmake supports Makefile on its list of file formats, and it then builds everything fine. If you wanted to build and install every contrib code module, you’d do that like this:
$ cd contrib
$ gmake
$ gmake install
Parser source rebuilds
One more related note here, while I’m mentioning build trivia. I mentioned flex and bison aren’t needed with official release source code archives. Building without them will give warnings like this:
configure: WARNING:
*** Without Bison you will not be able to build PostgreSQL from Git nor
*** change any of the parser definition files. You can obtain Bison from
*** a GNU mirror site. (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this, because the Bison
*** output is pre-generated.)
checking for flex... configure: WARNING:
*** The installed version of Flex, /usr/bin/flex, is too old to use with PostgreSQL.
*** Flex version 2.5.31 or later is required, but this is /usr/bin/flex version 2.5.4.
configure: WARNING:
*** The installed version of Flex, /usr/bin/lex, is too old to use with PostgreSQL.
*** Flex version 2.5.31 or later is required, but this is /usr/bin/lex version 2.5.4.
no
configure: WARNING:
*** Without Flex you will not be able to build PostgreSQL from Git nor
*** change any of the scanner definition files. You can obtain Flex from
*** a GNU mirror site. (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this because the Flex
*** output is pre-generated.)
But that’s fine; they are just warnings. And if you use make clean to clear that source code directory tree up, it won’t delete these files, and compiles will still be fine.
Aside: if you read these warnings carefully, you’ll see why I didn’t even try to compile PostgreSQL 9.3 here from a source code checkout, one without the parser files. The problem with this server isn’t that bison and flex are missing. It has versions of those tools that are too old. Missing packages are easy to add if you’re root. Packages that need a new version can be a much harder problem. I’ve done that before to update flex on RHEL, and that is no fun at all.
There are two other option available beyond this simplest clean though, using names similar to those available for gcc builds. There’s make distclean, which eliminates everything that configure generated. This is the right type of cleanup to use if you have added new software to the server and now you want to re-run configure to find them. It also works for cleaning up before changing the options passed to configure.
There’s a third clean option too:
make maintainer-clean
The goal of maintainer-clean is to eliminate everything that is generated by the source code build process. And I’d gotten into the habit of always using this one when I needed to eliminate a now obsolete configure session. But this is a bad idea if you’re trying to build PostgreSQL on a system without a full development toolset. maintainer-clean wipes out all of the parser files. And if those are gone, you’re back to where you need bison and flex again! Moral of this part: if you’re using a source code release archive and hoped to avoid bison/flex, don’t ever run maintainer-clean on the source code tree. That will blow things away so that what you get is more like a git checkout of the code. It won’t compile unless you have those parser generator tools. You may have to re-extract the original release distribution archive to get the parser code back again.
The post Making make make PostgreSQL 9.3 on FreeBSD 8.2 appeared first on High Performance PostgreSQL.