In this tutorial, we will dive deeply into OpenShift to understand the custom build and deployment process. We will also learn the command-line tool for logging and troubleshooting when our application is down.
We'll also cover some bonus tips to leverage the SSH features offered by OpenShift. Additionally, we'll look at how to synchronize our local environment with remote environment including database, source file and media files.
The first part of this series gave a quick overview of OpenShift. We discussed how to create the app, get the server URL, our account name, and setup a custom domain.
We did almost all of those tasks using the web interface which is great and very convenient; however, in addition to the dashboard, OpenShift offers a powerful client tool call rhc client. We
can invoke it from command line to perform OpenShift administration and maintenance. Once
you;ve installed the tool, you can create apps, add cartridges, and add gears quickly. It's a Swiss army knife. You may not need it but it's very handy.
Install OpenShift Command Line Client
The OpenShift document is very clear about installing this client library. The library is written in Ruby so make sure that you have Ruby installed. Basically, you only need to install the rhc gem on Mac or Linux. You've got a large chance that Git and Ruby are already installed so you only need to run:
Then setup it with your username and password. When you are asked to Generate a token now? Type Yes.
Now that we have the utility installed, let's play around with it.
The first thing to note is that the rhc command will give you a list of available commands. You can learn from there with rhc help. It shows a brief overview of each command. rhc help command_name will show you how to use a particular command.
For the command that interacts with an app, you have to specify the app name with -a appname or just append app name; however, if you run the commands inside your Git repository that you cloned before, you may omit it. The app name can be see with rhc app command. Example with my previous demo2 app. Instead of typing whole the account name and server name to SSH into it, I can use this command:
Sometime, it can be useful to trigger a deployment without any pushing. Say we want to deploy WordPress from a particular Git commit or a special branch.
See the following example:
Or show app information:
You can also save a snapshot of current app:
Or you can add some cartridges:
Of course, feel free to experiment with a number of different commands. It's relatively straight forward and easy to understand, and since everything is under source control, it's easy to rollback your changes.
What Happens When You Push to Your Repository?
In the first article, we saw that whenever a git push to deploy a branch is triggered, the app will be deployed. According to the OpenShift document, here is a break down of what happen:
You run a git push on your computer, your changes are sent to your OpenShift application
The application is shut down
Your changes are copied on the server into the correct location
OpenShift invokes your build hooks - script files you've placed in your Git repository
Your application is started
Step 4 is handle by script files in your .openshift/action_hooks. OpenShift will execute those script files checked into your Git repository at
specific points during the deployment process.
If the corresponding
script file does not exist, the deployment process will continue
normally. Again, all hooks must be placed in the .openshift/action_hooks/ directory in your application repository. The individual phases of each build are:
Pre-Receive. During your push, OpenShift checks to ensure that your application is in a consistent state. There is no hook for this step.
Pre-Build.
This happens after the application is stopped and the new repo dir has been deployed but before the build. Runs the .openshift/action_hooks/pre_build script
Build. This builds your application, downloads required dependency, executes the .openshift/action_hooks/build script and preps everything for deployment. In scope of WordPress, we don't use this step much since WordPress just need to drop into document root and run by web server, no any special build requirements. We usually use this hook to Download WordPress and extract it into correct location on OpenShift.
Deploy.
This step happens right before the application is issued a start. Any
required prep work to get the application ready to be started should be
done in the .openshift/action_hooks/deploy hook. In scope of WordPress, we use this hook to copy the data of the above build into document root, copy plugin and theme (.openshift/themes and .openshift/plugin) to the correct location.
Post-Deploy.
Some applications may need to interact with the running application to
complete the deployment process. After the application starts, the .openshift/action_hooks/post_deploy hook will be executed.
You can totally customize these scripts for your own purposes. The build scripts are executed directly, meaning you can write it in any language no matter it's Bash, Ruby or Python as long as you put correct shebang such as if you used bash.
Or Ruby
I prefer to write simple Bash script for build purpose and I will use it in this tutorial. Bash is every where and its syntax is so easy that you don't need to know Bash to understand some scripts actually.
With respect to WordPress, we don't do many tasks in the building process because PHP doesn't requires building; however, we utilize that build process to prepare for some data. The .openshift/action_hooks/build checks whether WordPress is created on OpenShift, if not it will download the WordPress from WordPress.org, create necessary directory, extract and copy content of the WordPress source files into the correct location. Since the second deployment, that build script does almost nothing because WordPress has been installed. To customize these build script, we have to know the environment variables.
The OpenShift Environment Variables
During our build and document process, we will need to know some OpenShift information such as the document root path, the data path, and the application name. This information is available in environmental variables for which a full list can be access here.
Some variables that we'll use frequently is as follows:
OPENSHIFT_HOMEDIR. home directory path. On your local machine, Linux you will have your user home directory in /home/username or on Mac OS X it's /Users/username. Home directory is the folder you will be in right there when you remote access via SSH. It's /var/lib/openshift/user_account_id. For example, my own is /var/lib/openshift/532bd7655004468bcf0000e1.
OPENSHIFT_REPO_DIR. $OPENSHIFT_HOMEDIR/app-root/runtime/repo/. Repository containing the currently deployed version (of the application). It's exactly the same with whatever you have inside your repository on your local machine with an exception that php folder now is symbolic link point to document root; therefore, $OPENSHIFT_REPO_DIR/php can be used in build script to reference to document root.
OPENSHIFT_DATA_DIR. $OPENSHIFT_HOMEDIR/app-root/data/
OPENSHIFT_APP_UUID. The unique id of your app on OpenShift. UUID can be very handy when you start to generate paths.
We access these variable in our build script with $variable_name.
The Custom Build and Deployment Process
Let's look back our current repository structure:
We'll start with a Hello World action so we have a sense of what will be output, when and how as we customize the build script.
Your Hello World Build
Let append this echo "Hello world. This is invoked before building." into your ./openshift/pre_build
Push to the repository and you will see the output like this:
As you see, our message appears during build process. Now, let's do a real thing to copy files during build process.
Copying File to Document Root
If you recall previous step, the file in .openshift/themes and .openshift/plugins is copied to wp-content/themes and wp-content/plugins. Let's say we want to do the same for copying file to the document root.
You want to create a sub folder call resume and put a file resume/my_resume.txt and make it access via demo2-tutsplus.rhc-cloud.com/resume/my_resume.txt. Let's create a directory to hold the data that we will copy into document root. We create a folder call docroot inside .openshift and whatever inside it will be copied.
Let's open .openshift/action_hooks/deploy, notice line 49 (I make it bold)
We added this command after that.
The line with # is comment, for our own reference. The echo line is just to show some output. The real command we used is cp to recursively copy all files and folders.
Now, let put something in .openshift/docroot and deploy.
Your Exercise
Use the php folder in your repository as a way to store this content, instead of introducing .openshift/docroot.
Hint
app-deployments/current/repo holds exactly same copy of your repository. Whatever you have on your local machine will be here.
Troubleshoot and Maintain
In the previous part, you knew how to remote access to your application with SSH. Once you are in, you can use Linux command.
However, rhc come with command ssh allow you to connect into an app. Quickly and easy to remember.
From these, you can always use help to show available commands.
tail_all: tail all of your log. You can see real time logging with this command for all gears in system: such as access log of Apache, MySQL error log.
mysql drops you into a MySQL shell. Very handy comparing with typing a long mysql command with host name, user string and password.
export show all current environment variables. During working with build script, you can use this command to see the list of available environment variables.
gear to control your gear: start, stop, restart. Like you can restart apache, stop mysql.
snapshot snapshot take a full backup of your current WordPress with all file, database dump, and media data. A very good way to have a full back-up of your site.
quota show your disk quota. Useful when you cannot upload to WordPress anymore, you may run out of space.
Taking a Snapshot
rhc snapshot-save demo2 dump your database, compress the media file, and source code, putting together a tar file and download it for you.
By default, a tar file with same name as your app is
created. Store it somewhere for your shake. Once you have the snapshot
saved, you can restore as well.
Checking Gear Status
The gear command controls cartridge status, start/stop. Like you get a timeout error when visiting your domain, or some 404, 503 error, or a database error. You have to SSH into your app and check gear status:
If it's saying CLIENT_RESULT Application is either stopped or inaccessible, I have to start it with:
The next step is to consult your log file.
Logging
tail_all show you completed logs of all cartridge in real time. However, if you want to look at an individual log, here is the list:
php/logs access and error log of Apache and MySQL
mysql/log MySQL log
Port Forwarding
The rhc port-forward command can help you establish a local
connection to your hosted service(Web server, Database server...). OpenShift automatically checks available ports on your local system and forwards one to a remote port of running service. Via port forwarding, you can work on local machine but the connection is forward to remote machine.
Let's try with MySQL.
I had port 8080 and 3306 for my different apps. Therefore, OpenShift picked up 3307 for MySQL. Now I can use the MySQL credential to connect to it with Sequel Pro. If you forgot your password, you can get it again on OpenShift dashboard or SSH into server, and issue:
Using Sequel Pro to connect to it.
Synchronize Local and Live Environments
Everybody loves to develop on a local machine instead of uploading to server for testing or evaluate a feature. You may export data from live host and import on local machine for that purpose like in this tutorial. However, that exporting process is annoying. The data can be big, take longer and longer to export/import. Fixing bug is harder, too. If a bug only happens on a particular post/page on live site, without re-producing the exactly post/page content, it can be hard to debug on local environment.
Or if you are a remote team, and you have your team created lots of test post on a staging/testing area (hosting on OpenShift), it will be good if you have a good way to just synchronize everything back, specially the pictures data.
Therefore, I propose a method to achieve that with: port forwarding and host file editing. Using port forwarding as we saw above, we can access MySQL easily from local machine. With the help of editing host file, we can point the domain openshift.axcoto.com to local machine, instead of to OpenShift. In our clone repository, we have an empty php folder. When deploying, that folder is replaced by a symbolic link to document root of Apache. Now, we will use that folder as document root on our local machine. Recall the repository structure again, on my computer.
Note that, to succeed at this method, you should add whatever you have
in wp-content/plugins(default WordPress 3.8.1 plugins) and
wp-content/themes(default WordPress 3.8.1 themes) on OpenShift into your
Git repository. You can download it via scp or SFTP as we discuss in the previous article, or you can simply download the default ones from WordPress.
Just make sure your local repository .openshift/themes and .openshift/plugins has same thing with the wp-content/themes and wp-content/plugins folder of WordPress app on OpenShift.
Step 1: Port Forwarding
Using command rhc port-forward demo2, with demo2 is my app. Change it to your app name.
Step 2: Virtual Host
Using Apache, we will add one more entry for our domain point to folder hold our WordPress. Depending on your OS, you may have different location for Apache configuration. It can be /etc/apache2 or /etc/httpd.
For example, on my Mac, it's /etc/apache2. I will add below code to my /etc/apache2/httpd.conf, or /etc/apache2/extra/httpd-vhosts.conf depend your OS, or where you prefer.
If you are not familiar with Apache and Virtual host, some article will helps:
http://code.tutsplus.com/articles/apache-2-basic-configuration-on-unix-like-systems--net-26607
http://code.tutsplus.com/articles/how-to-setup-a-wordpress-development-environment-for-windows--wp-2...
http://code.tutsplus.com/tutorials/wordpress-development-and-deployment-with-mamp-git-and-dropbox--w...
Step 3: Environment Variable
When running on your local
machine, you won't have the environment variable available. We have to
define in somewhere. For a simple starting, I prefer to put these
variable directly in Apache config file with SetEnv direction.
Our virtual host entry become:
I grab the environment variable of OpenShift and just put it into
our config file. But I change the MySQL host to 127.0.0.1 and MySQL Port
to 3307 because we do port forwarding before.
Step 4: Changing the Host File
openshift.axcoto.com is configured to point to OpenShift. However, we can override it by directly editing /etc/hosts. Open that file with you favorite editor and append
Confirm that it's pointing to our local machine.
If you want to visit the real app on OpenShift, comment out that line in /etc/hosts. Put a "#" to comment out:
Then openshift.axcoto.com will point to live site again.
Step 5: Pull WordPress Core File and Sync Media From Live Site
Finally, we have to copy the media and WordPress file into our local environment. We also make the symbolic link to point php/wp-content/themes to .openshift/theme. The same concept applies for plugin, php/wp-content/plugins point to .openshift/plugins. This way we can working on .openshift/themes and .openshift/plugins and can refresh browser to see our change.
The first time you want to sync the data, you run these commands; however, since the second time, we don't want to pull all of file because partial of files were there. A better option is only sync uploads file. We can do that with scp:
Or with rsync. Rsync is much better because it only download the not exist file. Same file won't be downloaded again.
You may go further, put these commands into a shell script and track it with your Git repository too. I will leave that part for you :)
You may want to ignore file inside php because those file are just for our own testing/running on local computer. No need to track them.
Step 6: Administration Over SSL (Optional)
The wp-config.php, that OpenShift generated, defined to serve administrator over SSL
On your computer, you usually don't have SSL certificate. You may edit wp-config.php and set FORCE_SSL_ADMIN to false. If you want challenge, and want to give it a try to run a SSL connection then here is the instruction.
Generate SSL certificate
This is self-sign certificate meaning that browser won't trust it. It's fine because we are working on local anyway and you can just add an exception when the browser throw an untrusted warning.
We create the folder /etc/apache2/ssl, then generate the cert and key into that folder. We will config apache point to these two files later.
Define Virtual Host Entry
Open your /etc/apache/httpd.conf (again, your Apache config file can be in different folder, such as /etc/httpd), find the line Listen 80 and append Listen 443
On top of your /etc/apache2/extra/httpd-vhost.conf, find the line NameVirtualHost *:80 and append
Next, we define one more entry in virtual host for port 443. We just duplicate the previous definition with port 80, but this time, we add some SSL definition.
Note the SSL certificate file path. Restart Apache with sudo apachectl restart. Now, you can access your dashboard over SSL. Here is my WordPress installation, running totally local with same database from live site, and media data is synced.
Conclusion
Throughout this series, we gained a lot of knowledge about OpenShift. You should not be able to easily install and configure WordPress on OpenShift. With the handy of rhc client, you can quickly config or viewing app information from terminal.
We also covered build and deploy scripts so you can customize your application to your needs like showing a special banner, and a push to HipChat during deployment. You can even sync data between your local computer and OpenShift. I hope you enjoy this tutorial and OpenShift.
Leaving comment to let us know how you are doing. If you run into trouble, let me know in the comment, I would like to help.