Deploying and maintain a Drupal site from a development PC

Recently, I deployed a site created on my development machine to the InterWebs. I wrote down what I did. Here are the notes.

Important! There may be better ways to do things. This worked for me, on my setup. YMMV, and other warnings.

In the beginning:

  • Windows 7 PC.
  • IDE: Netbeans (PHP debugging, and Git support).
  • Xampp - a Drupal/MySQL/Apache stack on my development machine.
  • Drupal 7.
  • Bunch o' contrib modules.
  • Bunch o' custom modules.
  • Python code that Drupal PHP calls. Needs Python's docutils library.
  • Contrib theme - Bootstrap.
  • Custom subtheme based on Bootstrap.
  • MySQL database.
  • Many nodes already in the database.
  • Bunch o' uploaded files to go along with the nodes (images and such).
  • All project files on GitHub, except for settings.php and the contents of the sites/default/files directory.

Deploying the site to a HostGator VPS.

Git on dev machine

NetBeans has GUI Git features built-in. I'm no Git expert, but I've found that using it makes deploying files easier. Much easier.

I told NetBeans to create a Git repo for the project. Right-click on the project, Versioning | Create Git repository.

GitHub

I made a repo for the project on GitHub. A private account costs a few dollars per month - well worth it.

There are some files that should be different betwixt dev and live. I didn't want those to be version controlled. I created a file called .gitignore in the root of the local project (also the root of the repo). It tells Git to ignore the following:

  • Files uploaded by users. They are in sites/default/files.
  • The database settings file, sites/default/settings.php. The name of the database, user, and password are different on dev and live.
  • Metadata files created by NetBeans. They are in the directory nbproject.

So the .gitignore file was:

sites/*/settings*.php
sites/*/files
sites/*/private
/nbproject

GitHub gave me the URL that NetBeans should use to push changes to GitHub. I copied-and-pasted that into NetBeans.

Finally, I told NetBeans to push all the code to the remote repo. All of the NetBeans work is GUI - yay!

Get a domain

I registered a domain through HostGator. Easy peasy. The domain was dyjbit.com. Stands for Do Your Job Better with IT.

Create a home on the VPS

HostGator gave me Parallel Panels. It's a control panel for administering your Web site, like cPanel, but designed for resellers, VPSers, and others. I don't grok it very well, but enough to get by.

I created a Webspace (a Parallel Panels concept) for the site. That creates a directory, an FTP/SCP user account, and some DNS settings. The directory was /var/www/vhosts/dyjbit.com. I got to name the user, so I called it... let's say, frankftp.

Pro tip: if you have Parallel Panels or cPanel or some other control panel, let it create the Web space for you, including the FTP user. Drupal file permissions have to be set in a certain way. The control panel will get most of that right. More later.

DNS

Next, I used a HostGator tool at https://register.hostgator.com/ to specify the nameservers. They are private nameservers on the VPS.

It took the full two days to propagate. For one of those days, I could ping dyjbit.com on my phone (an Android phone with a terminal app), but not from my dev machine. Got there eventually, though.

Git (most of) the files

The files were already on GitHub, so I decided to just grab them from there.

First, I terminaled to the VPS with PuTTY. I logged in as frankftp. Don't use root, or some other account. Use the account your control panel created.

Pro tip: set a keep alive value in your terminal program, so that the session doesn't time out.

Next, I changed to the Drupal root directory (/var/www/vhosts/dyjbit.com/httpdocs), and initialized a repo:

git init

Then I connected that server repo to GitHub:

git remote add origin https://github.com/[user name]/[repo name].git

Finally, I grabbed the files from GitHub:

git pull origin master

Why keep the files on GitHub? Because deploying code changes is easy. This site has lots of custom code, and lots of custom bugs. Here's how I deploy a code change:

  • Update the code with Netbeans (my IDE of choice) on my dev machine.
  • Commit the changes on the local dev machine, with NetBeans' handy dandy Git GUI.
  • Again with NetBeans, push the changes to the project's repo on GitHub. I just use the master branch, because I don't know any better.
  • Start a terminal program (PuTTY or whatever). Login to the server, using the FTP account created by the control panel (Parallels Panel, cPanel, etc.). frankftp in my case.
  • cd to the Drupal site.
  • Tell Git to grab the most recent changes from the GitHub repo: git pull origin master

You don't need to keep track of which files have changed. Git does that. Hooray!

The rest of the files

Not all of the files were in Git. I used WinSCP (FTP client) to upload the contents of sites/default/files - user uploaded files. On the server, I copied default.settings.php to settings.php, for later editing. I used the frankftp user to do that.

Pro tip: tell WinSCP to refresh every so often, so the connection isn't lost.

The database

I used phpMyAdmin on my dev machine to export the database to a zipped file.

Then I created a new MySQL database on the server, using the Paralles Panels control panel thing. The name of the database on the server was different from the name on dev; that's common. I created a MySQL user with a password that could access the database.

phpMyAdmin is on the server as well. Used it to import the zip file exported by phpMyAdmin on the dev computer.

Updating database connection information

The information Drupal uses to connect to the database is in sites/default/settings.php. I edited that file on the server. I used the editor built in to WinSCP (the FTP program I use). I could have used nano through the terminal, but the WinSCP editor is easier.

Pro tip: I have WinSCP linked to Notepad++. So when I tell WinSCP I want to edit, it starts Notepad++ and sends the file there.

The connection information is like this:

$databases = array (
  'default' =>
  array (
    'default' =>
    array (
      'database' => 'xxxx',
      'username' => 'xxxx',
      'password' => 'xxxx',
      'host' => 'localhost',
      'port' => '',
      'driver' => 'mysql',
      'prefix' => '',
    ),
  ),
);

Replace the xxxxs with what you specified on your server.

Python

One of my modules calls custom Python code. That code needs Python's docutils library.

Python is already installed on most servers, but I needed to make sure. I wrote a short test program called test.py:

def main():
    print(1+1)
    pass

if __name__ == '__main__':
    main()

To see it if it runs:

python test.py

It output:

2

Yay! Python was running.

What about the docutils library? Was that installed? Another test program, called test_docutils.py:

import sys
from docutils import nodes, core, io
from docutils.parsers.rst import Directive, directives
from docutils.parsers.rst.roles import set_classes
from docutils.utils.code_analyzer import Lexer, LexerError, NumberLines

print 'YAY!'

It ran, and had the expected output. So the library was there. If it hadn't been, I would have asked my old friend Google how to install it.

The Python I wrote uses the reStructuredText code in docutils. Would that work? Yet another test program, test_rest.py:

import sys
from docutils import nodes, core, io
from docutils.parsers.rst import Directive, directives
from docutils.parsers.rst.roles import set_classes
from docutils.utils.code_analyzer import Lexer, LexerError, NumberLines

class Exercise(Directive):
    has_content = False
    required_arguments = 1

    def run(self):
        result = '[[[cycoexercise ' + self.arguments[0] + ']]]\n'
        raw_node = nodes.raw('', result, format='html')
        return [raw_node]

#Register the new directives.
directives.register_directive('exercise', Exercise)

#Read the content to translate.
f = open('test_rest.txt')
content = f.readlines()
f.close()

data_in = '\n'.join(content)

#Parse some content.
doc = core.publish_parts(data_in, writer_name='html')['html_body']
print (doc)

It reads the data file test_rest.txt:

Test
====

This is a *test*.

.. exercise:: 666

This is nothing.

Ran the program:

python test_rest.py

The output:

<div class="document">
<p>Test</p>
<hr class="docutils" />
<p>This is a <em>test</em>.</p>
[[[cycoexercise 666]]]
<p>This is nothing.</p>
</div>

This is as expected. Yay!

Trying the home page

I went to the home page of the new site: http://dyjbit.com. It showed up! Yay!

Well, yayish.

I knew from experience that I'd need to do more stuff to get the file system working right. To begin, I turned off write permissions to settings.php, for everyone, including the file owner.

Then I want to admin/config/media/file-system, expecting errors. There they were! I had to do two things.

First, change the directory for temporary files. The dev machine runs Windows, and used the file path D:\xampp\tmp. That makes no sense to the Linux VPS. So I changed it to /tmp, that's standard on every *nix machine I've worked with.

Second, I needed to change the permissions for sites/default/files directory, and its subdirectories. The control panel had set things up so that only the file owner had write permissions. The file owner is the user that the control panel created, frankftp, and that's mostly right. But Apache needs write access to that directory too, since it will store files there that users upload. Apache runs under its own user account, called apache, nobody, or something else.

What to do? *nix files have two owners: a user, and a group. The control panel (Parallels Panels) put frankftp in the same group as the Apache user. So, all I had to do was give write permissions on sites/default/files to members of the group. I did that with WinSCP (the FTP program I use, remember). I right-clicked on the directory (files), and chose Properties. Then I checked Group Write, checked Apply recursively to descendants, and clicked OK.

(You can read more about Drupal, Apache, and Unix permissions.)

Install Drush on the server

Here's how.

Note! Take care when updating modules with Drush on the server. Actually, best not to do it, since your repos will get out of sync. More below.

Fix random broken things

There are always problems. Things you don't anticipate. Grunge left over from past sysadmins (that would be me).

Cache clearing and WSOD

First time I cleared the Drupal cache - WSOD! That's white screen of death.

The PHP memory limit defaulted to 96M. I set it to 768M, in the control panel. The problem went away.

Why 768? That's the maximum in my VPS service plan.

Taking notes

I like to create text files with documentation, reminders, commands to cut-and-paste, etc. The site file root is /var/www/vhosts/dyjbit.com/httpdoc. My notes files go in its parent, /var/www/vhosts/dyjbit.com, so the notes aren't accessible o'er the Web.

So, I logged in as frankftp, went to the right directory, and tried to create a notes file. Argh! I couldn't. The user I was, er, using, frankftp, didn't have permission to write to the directory. It turned out that /var/www/vhosts/dyjbit.com was owned by root. I could have added the notes files by logging is as root, but that didn't make sense to me. It made more sense for frankftp to be able to make notes.

To fix it, I logged in as root (for one command), and used chmod to give the FTP user ownership of /var/www/vhosts/dyjbit.com. Then frankftp could add notes files.

Installing new modules

Changes to my custom code are deployed as described above. What if I want to install a module? That's a little more complex.

When you add a new module to your site, there are two (at least) types of changes:

  • Adding new files, usually under sites/all/modules.
  • Changing the database.

The two have to be done separately.

First, I use Drush to download the module to my dev machine. I do not install the module - yet. Just download the files.

Important! Do not use Drush (or FTP) to add the files to your server. You run the risk of one of your Git repos getting out of sync with your server.

Second, I Git commit on the dev machine. That puts the new module's files into the local repo.

Third, again on the dev machine, I push the changes to GitHub. That will copy the new module's files into the GitHub repo.

Fourth, I log in to the server with the FTP account that the control panel created: frankftp. Not as root. If you log in as root, you might mess up some file permissions.

Fifth, I switch to the Drupal root directory, and git pull origin master on the server. That pulls the new module's files down from GitHub.

Then I can turn on the module in Drupal.

Updating a module (from contrib - not one of your own)

Modules updates typically affect:

  • The code base - replacing files in the module's directory.
  • The database - update code in the module changes the database.

First, use Drush to update the module on your development machine. Not the server!

drush up [module]

This will update the code base and the database locally.

Second, commit the local changes, and push them upstream.

Third, login to the server, and pull the changes down:

 git pull origin master

Fourth, run the database updates. You can see what database updates are pending with this command:

drush updatedb-status

Run the updates like so:

drush updatedb

Drush will ask for confirmation.

That's all, folks!

Hmm. It can be harder than you think.