vm_making_of

Our Virtual Machine - the "Making of"

Author: Wojciech Rygielski - [email protected]

Introduction in Polish

To są wszystkie kroki jakie wykonałem podczas tworzenia wirutalnej maszyny udostępnionej na stronie projektu OC. Raczej nie potrzebujesz tutaj zaglądać, chyba że zamierzasz wypuścić nową wersję maszyny - wtedy ten dokument należy uzupełnić. Może też być przydatny dla kogoś, kto chce zainstalować OC na fizycznej maszynie (zamiast VM).

Introduction in English

These are the steps I did to setup [our virtual machine] (vm_installation_and_usage). I write this in English because it might come in handy for anyone trying to setup OCPL on a physical server (i.e. new OC nodes). So, this might be considered as an unofficial installation guide.

Steps

Version 1 (2012)

# First, I downloaded VMware Ubuntu image from:
# http://linhost.info/downloads/Ubuntu-11-04-Server-32-bit.7z

# Logged in, and executed:

sudo rm /etc/udev/rules.d/70-persistent-net.rules    # to get the network running
sudo reboot
sudo apt-get update                 # to refresh the list of packages
sudo tasksel install lamp-server    # apache, php, mysql
sudo tasksel install openssh-server # for tools like putty and winscp
sudo apt-get install mc             # midnight commander
sudo apt-get install subversion     # this got me svn 1.6
sudo apt-get upgrade                # I was hoping to get svn 1.7 with that, but no ;>
sudo apt-get dist-upgrade

# At this point I still had svn 1.6 (you may check yours with "svn --version").
# I wanted 1.7, so I did the following. This is an unofficial package of svn 1.7.
# If you prefer to stick with 1.6 then just skip these steps.

sudo apt-get install python-software-properties
sudo apt-add-repository ppa:dominik-stadler/subversion-1.7
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade

# Then I changed the server name to "ocpl-devel" (edited /etc/hostname
# AND /etc/hosts files). You can change it to whatever you like.
sudo reboot         # To get the new name working
sudo apt-get clean  # Removed old packages

# Checked out OCPL code by root, then changed the owner.
sudo svn co https://opencaching-pl.googlecode.com/svn/trunk/ /srv/ocpl
sudo chown -R user /srv/ocpl
sudo mkdir /srv/ocpl-dynamic-files               # for dynamic files generated by our scripts
sudo chown -R www-data /srv/ocpl-dynamic-files/  # 'www-data' is the Apache's user

# Time to create OC settings file.

cp /srv/ocpl/lib/settings-example.inc.php /srv/ocpl/lib/settings.inc.php
vi /srv/ocpl/lib/settings.inc.php

# Changed domain name to 192.168.163.130 - this was the IP address which
# VMware generated for my VM. Get your IP by running 'ifconfig'. Changed
# lots of other settings (database name, password; email addresses etc.)
# All errors and messages should be now sent to user 'user'. Run 'alpine'
# to read them.

# Connecting our installation to Apache.
sudo vi /etc/apache2/sites-available/default

# I put something like this inside:
<VirtualHost *:80>
        DocumentRoot /srv/ocpl
        Options -Indexes

        Alias /images/uploads/ /srv/ocpl-dynamic-files/images/uploads/
        Alias /uploads/ /srv/ocpl-dynamic-files/images/uploads/
        Alias /upload/ /srv/ocpl-dynamic-files/images/uploads/
        Alias /wigo/ /srv/ocpl-dynamic-files/wigo/
        Alias /download/ /srv/ocpl-dynamic-files/download/
        Alias /images/statpics/ /srv/ocpl-dynamic-files/images/statpics/
        Alias /images/mini-mapa/ /srv/ocpl-dynamic-files/images/mini-mapa/
        Alias /mp3/ /srv/ocpl-dynamic-files/mp3/

        ErrorLog ${APACHE_LOG_DIR}/error.log
        1. Possible values include: debug, info, notice, warn, error, crit,
        1. alert, emerg.
        LogLevel warn
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# Mods and database setup.

sudo vi /etc/logrotate.d/apache2  # changed 640 to 644, to make log files readable by user
sudo a2enmod rewrite              # this mod is required, it wasn't enabled by default
sudo service apache2 restart
mysql -uroot -ptoor
create database ocpl default character set utf8 collate utf8_general_ci;
grant all on ocpl.* to user@localhost identified by 'ubuntu';

# Loaded a sample database. It is based on the original production DB, but
# with all private data removed. Also, the number of logs was reduced.
# Some other tables also have been truncated (to keep the DB small and safe).
# Most OC tables do not use any kind of structure versioning (only okapi_*
# tables do - run "svn up" and "wget -O - -q http://localhost/okapi/update"
# to update OKAPI tables to the newest revision).

# Created default directory structure in /srv/ocpl-dynamic-files. I found this
# structure here: /srv/ocpl/docs/ocpl-data.tar.

# Noticed that some pages require GD.
sudo apt-get install php5-gd

# (published version 1 at this point - 814 MB, md5: 01e4ef90dd6cf5d7e2a7400390163e1a)

Important: You will probably need to go though all the steps up to the highest version before you will be able to run current version of OCPL code from SVN. For example, since ~August 2012 OCPL requires PHP's APC module which was added to our VM in version 5.

Version 2 (2012)

Version 2 (349 MB, md5: 89d432fffb584df02705ac6027e3771f) was reverted. Version 3 is based on version 1 instead.

Version 3 (2012)

# Added an email client for reading oc-admin mailing delivered for the user "user".
sudo apt-get install alpine

# Then I remembered about the cronjobs. Most of them are not necessary in a
# devel environment. So I added OKAPI's cronjobs only. If you ever need to
# add more, see /srv/ocpl/docs/sql/cron-job-list (if not there, look else
# in the docs directory).

crontab -e    # as user 'user'
# I added the following line:
*/5  *  *   *   *   wget -O - -q -t 1 http://localhost/okapi/cron5

# Some developers said they need this. I set the password to 'phpmyadmin'
# database to 'toor'. You should login via your usual credentials
# (username: user, password: ubuntu).
sudo apt-get install phpmyadmin

# This is a small and handy tool for measuring pipes.
sudo apt-get install pv

# Some other packages.
sudo apt-get install php5-tidy

# Updated the database structure. Now it is the same as it was in OC.PL on 2012-03-22.
# OC.PL still doesn't have versioning. You can use http://opencaching.pl/okapi/devel/dbstruct
# to compare the current database structure with yours. Currently there is one difference,
# triggers at OC.PL are installed on root, on the devel machine as 'user'.

# Added a script called "shrink-and-prepare.sh" to the user's home directory. In the
# next version it will be replaced with the one in repository (`trunk/docs/vm/shrink_and_prepare.sh`).

# (published version 3 at this point - 428 MB, md5: 5d2dcf49826f11cf434ed40b53c0fbce)

Version 4 (2012)

# Installed more entries in the `user`'s crontab. Currently crontab looks like this:

*/30 *  *   *   *   wget -O - -q http://localhost/tpl/stdstyle/etc/write_newcaches.inc.php
*/55 *  *   *   *   wget -O - -q http://localhost/tpl/stdstyle/etc/write_newblogs.inc.php
*/14 *  *   *   *   wget -O - -q http://localhost/tpl/stdstyle/etc/write_onlusers.inc.php
*/5  *  *   *   *   wget -O - -q http://localhost/tpl/stdstyle/etc/write_totalstats.inc.php
*/5  *  *   *   *   wget -O - -q -t 1 http://localhost/okapi/cron5

# I want to make maps v3 work. I will have to compile the mapper.fcgi file in order to
# achieve that. (EDIT 2014: mapper.fcgi is no longer used in maps v3,
# you may skip most of these steps!)

sudo apt-get install make
sudo apt-get install gcc
sudo apt-get install libsdl1.2-dev
sudo apt-get install libfcgi-dev
sudo apt-get install libsdl-image1.2-dev
sudo apt-get install libsdl-gfx1.2-dev
sudo apt-get install libsdl-ttf2.0-dev;
sudo apt-get install libmysqlclient-dev
sudo apt-get install libapache2-mod-fcgid
sudo a2enmod fcgid # just in case it wasn't already enabled

# Compared enabled mods with the production site. Moved most of the differences.
sudo a2enmod expires
sudo a2enmod headers
sudo a2dismod reqtimeout
sudo service apache2 restart

# Turned out, that map v3 requires higher version of SSL_ttf than the one
# I installed above with apt-get. I had to compile it from sources:

cd # return to home directory
wget http://www.libsdl.org/projects/SDL_ttf/release/SDL_ttf-2.0.11.zip
sudo apt-get install unzip
unzip SDL_ttf-2.0.11.zip
cd SDL_ttf-2.0.11
./configure
make
sudo make install

# Cleaned up. Removed ~/SDL_ttf_*.
# In /etc/ld.so.conf, added a line "/usr/local/lib".
# Then, I uninstalled the previous version (for the linker to see the new one).
sudo apt-get remove libsdl-ttf2.0-dev
sudo apt-get remove libsdl-ttf2.0

cd /srv/ocpl/lib/cgi-bin/
make -f Makefile.fcgi

# Compiled successfully! But still didn't work. Then I remembered to add
# data/mapper.ini file.
cp data/mapper.ini-example data/mapper.ini
vi data/mapper.ini # changed user/password data to ours
# Hopefully, in future versions of OCPL, there will be ONLY ONE configuration
# file. Now there are TWO. Anyway, after this step mapper began to work.

# Many users use it, so I added it. (Install this on development machine only.)
sudo apt-get install samba

# In /etc/samba/smb.conf:
# - removed the comment before "security = user",
# - added the following to the end:
[srv]
   comment = OCPL-DEVEL-SRV
   path = /srv
   browseable = yes
   writable = yes
   create mode = 0777
   directory mode = 0777

sudo smbpasswd -a user  # and set the standard 'ubuntu' password.
sudo service smbd restart
sudo service nmbd restart

sudo dpkg-reconfigure tzdata # an chose "Europe/Warsaw"

# Changed ~/shrink_and_prepare.sh script to use the freshest version of itself.
echo "svn up /srv/ocpl; /srv/ocpl/docs/vm/shrink_and_prepare.sh" > ~/shrink_and_prepare.sh

# (published version 4 at this point - 495 MB, md5: d42a49b4c5770007d9eb910706dfd451)

Version 5 (2012)

# Made a release upgrade (up to Ubuntu 11 release).
do-release-upgrade

# Note: This upgrade broke the old v3 fcgi mapper binaries, but they are
# currently not used. I tried to fix them, but this didn't work:
cd /srv/ocpl/lib/cgi-bin/
make -f Makefile.fcgi
sudo service apache2 restart

# Fixed error in lib/settings.inc.php. It contained invalid key in OKAPI configuration.
# OKAPI locale-specific pages displayed in English instead of Polish (see $OKAPI_SETTINGS['SITELANG'](srv])).

# Installed some locales. These are required (or will be required shortly) by OKAPI's
# internationalization support. OKAPI uses native PHP's gettext implementation.

sudo apt-get install language-pack-pl-base
sudo apt-get install language-pack-de-base
sudo apt-get install language-pack-se-base
sudo apt-get install language-pack-nl-base
sudo apt-get install language-pack-it-base
sudo apt-get install language-pack-es-base
sudo apt-get install language-pack-cs-base
sudo apt-get install language-pack-sk-base
sudo dpkg-reconfigure locales

sudo apt-get install secure-delete # for wiping swap partition

# Changed all http://192.168.*.*/ occurances in the settings.inc.php file to http://local.opencaching.pl/.

# Got rid of a warning displayed on VM start (see [here](http://finster.co.uk/2010/11/16/virtualbox-piix4_smbus-error/)).
# Added "blacklist i2c_piix4" to /etc/modprobe.d/blacklist.conf file. And then:
sudo update-initramfs -u -k all

# Added "Alias /tmp/ /srv/ocpl-dynamic-files/tmp/" to Apache configuration (/etc/apache2/sites-available/default).

# Some OC code is designed to work in a subdomain, specifically http://m.ocpl-devel/ for mobile
# version of the page. Reconfigured Apache to support this:
cd /etc/apache2/sites-available
sudo vi mobile

# I did put the following inside "mobile" file:

<VirtualHost *:80>
        ServerName m.ocpl-devel
        ServerAlias local.m.opencaching.pl

        DocumentRoot /srv/ocpl/mobile
        Options -Indexes

        ErrorLog ${APACHE_LOG_DIR}/error.log
        LogLevel warn
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# Make the new config file work.
sudo a2ensite mobile
sudo service apache2 reload

# Added "ServerName ocpl-devel" and "ServerAlias local.opencaching.pl" to
# Apache configuration (/etc/apache2/sites-available/default).

sudo vi /etc/hosts
# On the end of the 127.0.1.1 line, added "local.opencaching.pl" and
# "local.m.opencaching.pl" domains.

# Note: if you want to access the mobile version of OC you need to modify hosts file
# on the HOST machine. See VM installation instructions for more information.

# Installed more packages required by OCPL.
sudo apt-get install php-apc
sudo apt-get install gpsbabel
sudo service apache2 restart

# Changed ~/shrink_and_prepare.sh script to execute the freshest version of itself
# even when the file from svn does not have execution permissions.
echo "svn up /srv/ocpl; bash /srv/ocpl/docs/vm/shrink_and_prepare.sh" > ~/shrink_and_prepare.sh
rm ~/shrink-and-prepare.sh
chmod 700 ~/shrink_and_prepare.sh

# Fixed a mistake in settings.inc.php file (a variable called $ocWP was renamed to
# $oc_waypoint.

# Made one more release upgrade (up to Ubuntu 12 release).
do-release-upgrade

# (published version 5 at this point - 795 MB, md5: 887aa550ef2aea3873cfa76378866611)

Version 6 (2016)

January 2016. Fetched a "fresh" (and terribly old!) version 5 from the server. Checked the md5 checksum.

First, we needed a couple of release upgrades!

sudo do-release-upgrade

Tons of packages were downloaded. Two cnf files generated conflicts which needed to be resolved. This got me Ubuntu 14.04 LTS (16.04 LTS would be available in April 2016, but it wasn't back then).

The site stopped working after upgrade, got HTTP 403 at root URL. This was due to Apache being upgraded (link).

cd /etc/apache2/conf.d
sudo mv phpmyadmin.conf ../conf-available/
sudo a2enconf phpmyadmin
cd ..
sudo rmdir conf.d
cd sites-available
sudo mv mobile mobile.conf
cd ..
sudo rm sites-enabled/mobile
sudo a2ensite mobile
sudo vi sites-enabled/000-default.conf

Added this to the end (must have been lost during the upgrade):

<Directory /srv/ocpl>
    Require all granted
    AllowOverride all
</Directory>

sudo service apache2 restart

Site started to work again.

OCPL switched repositories since 2012. SVN is no longer needed, we need Git.

cd /srv/ocpl
svn upgrade
svn status --no-ignore

This got me the list of untracked files. Only the lib/settings.inc.php file seemed important. Stored it away.

mv lib/settings.inc.php ~
sudo apt-get remove subversion
sudo apt-get install git
sudo apt-get autoremove
rm -fR *
rm .htaccess
git clone https://github.com/opencaching/opencaching-pl.git .
mv ~/settings.inc.php lib/

• Updated the database, based on the OKAPI diff. (There are some differences, but they seem not so important.) Also updated OKAPI itself.

• Updated the settings.inc.php file, based on a copy from production site.

• Updated the ~/shrink_and_prepare.sh script to use Git instead of SVN in order to update itself.

Didn't have any more time for more fixes and upgrades...

Published version 6 at this point - 0.99 GB, md5: 30f69c68358f14e648140d78e2c6ebb2.

Version 7 (2016)

May 2016. Since opencaching.pl is switching servers, lots of upgrades will be performed. The new server will be using Ubuntu 16.04 LTS, so we need our VM to use it too.

This version is based on version 6. (Verified the md5 checksum before continuing and it matched.)

sudo su -
apt-get update
apt-get upgrade

This last command took so much time that I decided to bump the VM RAM at this point. It was set to 256MB, I have increased it to 1GB (prior shutdown was required).

sudo su -
do-release-upgrade -d

There were some questions during this update:

• I set MySQL server hostname for phpmyadmin to "localhost".
• Replaced configuration files with package maintainers' versions.
• Phpmyadmin threw "Empty value for 'port' specified" warning during the upgrade. At first I thought this was an error, so I retried and changed connection method to tcp/ip (default port), but it didn't help. Then I noticed that this was during a "backup phase", and that it was a warning (and can be ignored).

Then I got this...

Errors were encountered while processing:
 mysql-server-5.7
 mysql-server
Error in function:


*** Send problem report to the developers?

After the problem report has been sent, please fill out the form in the
automatically opened web browser.

What would you like to do? Your options are:
  S: Send report (533.3 KB)
  V: View report
  K: Keep report file for sending later or copying to somewhere else
  I: Cancel and ignore future crashes of this program version
  C: Cancel
Please choose (S/V/K/I/C):

This error is described here, but I have ignored it for now.

reboot

I lost network connectivity (screenshot, screenshot! Upgraded by VMware Player first, didn't help. ifconfig -a showed that the primary network interface name has been changed from eth0 to ens33. Tried numerous approaches with grub, but I couldn't bring eth0 back. So, eventually I did this:

vi /etc/network/interfaces

Replaced all occurances of eth0 with ens33.

reboot
ifconfig -a

We're back! IP address changed, so I had to update my host's hosts file.

sudo su -
apt-get autoremove
apt-get autoclean

Tried to run OC, but got Call to undefined function mb_internal_encoding() in error logs.

apt-get install php7.0-mbstring
service apache2 restart
cd /srv/ocpl
git pull --rebase

Removed ONLY_FULL_GROUP_BY from the current @@sql_mode:

/etc/mysql/mysql.conf.d/mysqld_set_modes.cnf

Pasted this:

[mysqld]
sql-mode="STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

service mysql restart

Updated /home/user/shrink_and_prepare.sh:

cd /srv/ocpl
sudo git pull
/srv/ocpl/docs/vm/shrink_and_prepare.sh

Published version 7 at this point - 0.99 GB, md5: 30f69c68358f14e648140d78e2c6ebb2.

Important: VM is now running PHP 7.0.4 and MySQL 5.7.12. At the time of writing this, parts of OCPL code are still not compatible with this setup. This VM version is supposed to be used interchangeably with version 6, for upgrading OCPL code to PHP 7.

Important note by Harrie Klomp Because statistics are not working in PHP 7.0.4 it is best to downgrade the PHP version to PHP 5.6. See description on http://askubuntu.com/questions/761713/how-can-i-downgrade-from-php-7-to-php-5-6-on-ubuntu-16-04.

Requests for the next VM version

Publishing new VM version

These notes are just for myself (because I guess no one else would want to publish a new VM version). You should probably just skip this section.

Over fast network connection

1. ocpl-devel: Use shrink_and_prepare.sh.
2. ocpl-devel: Shutdown (sudo shutdown -P 0).
3. windows: shrink the virtual disk - If you use VMware (much better shrinking):
   • I used to use vmware-vdiskmanager.exe -k ocpl-devel.vmdk, but there is no such tool in recent versions of VMware Player.
   • Recently, I used the "Defragment" and "Compact" buttons in Virtual Machine Settings -> Hardware -> Hard Disk section of VMware Player. - If you use VirtualBox: vboxmanage.exe clonehd ocpl-devel.vmdk ocpl-devel-new.vmdk, then replace the original one with the new one.
4. windows: remove garbage from ocpl-devel directory, only ocpl-devel.vm* files should stay.
5. windows: compress the ocpl-devel folder with 7-Zip, upload to ocpl
6. windows: generate a short url (for the stats!) for http://opencaching.pl/download/ocpl-devel-r<N+1>.7z (http://goo.gl/)
7. windows: publish the new version of the machine on the vm_installation_and_usage wiki page.

Over slow network connection

1. ocpl-devel: Use shrink-and-prepare.sh.
2. ocpl-devel: Shutdown (sudo shutdown -P 0).
3. windows: shrink the virtual disk - If you use VMware (much better shrinking): vmware-vdiskmanager.exe -k ocpl-devel.vmdk - If you use VirtualBox: vboxmanage.exe clonehd ocpl-devel.vmdk ocpl-devel-new.vmdk, then replace the original one with the new one.
4. windows: copy ocpl-devel directory to ocpl-devel-r<N+1>
5. windows: xdelta3.exe -e -s ocpl-devel-r<N>/ocpl-devel.vmdk ocpl-devel-r<N+1>/ocpl-devel.vmdk vmdk-delta-r<N>-r<N+1>
6. windows: compress vmdk-delta... with 7-Zip, upload to ocpl
7. ocpl: cd ~/vm
8. ocpl: check if ~/vm/ocpl-devel exists. If not, uncompress the last published machine from /var/www/ocpl-data/download there (7zr x ocpl-devel-r<N+1>.7z)
9. ocpl: 7zr x vmdk-delta-r<N>-r<N+1>.7z
10. ocpl: xdelta3 -d -s ocpl-devel/ocpl-devel.vmdk vmdk-delta-r<N>-r<N+1> ocpl-devel/ocpl-devel.vmdk.new
11. You may want to check if the checksum of vmdk on your windows matches the one on ocpl.
12. ocpl: mv ocpl-devel/ocpl-devel.vmdk.new ocpl-devel/ocpl-devel.vmdk
13. ocpl: 7zr a ocpl-devel-r<N+1>.7z ocpl-devel
14. ocpl: sudo mv ocpl-devel-r<N+1>.7z /var/www/ocpl-data/download
15. windows: remove ocpl-devel<N> (no longer necessary).
16. windows: generate a short url (for the stats!) for http://opencaching.pl/download/ocpl-devel-r<N+1>.7z (http://goo.gl/)
17. windows: publish the new version of the machine on the vm_installation_and_usage wiki page.