Changes

Page history
Consolidate Tech Working Group wiki authored by Ed Summers's avatar Ed Summers 
This commit moves information from the Technical Working Group's (TWG)
[git.coop
wiki](https://git.coop/social.coop/tech/operations/-/wikis/home) into a
subdirectory of the main wiki. The current TWG wiki on git.coop is
currently public. These changes are predicated on the idea that we want
to move forward with the
[proposal](https://www.loomio.com/d/2Rwh5z3H/time-to-consolidate-wikis-/69)
to consolidate wikis in preparation for the move to a more user friendly
wiki platform.

**Note:** This commit includes an update to the Tech Working Group section of the
Bylaws to point to the relevant section in the general wiki rather than
the old TWG specific git.coop wiki.

Quite a bit of the information contained in these pages needs to be
consolidated and/or updated. But for historical purposes the thought was
to bring these pages over largely unchanged and then update them either
here in Git, or once the content has been moved to a new wiki.

If/when this merge request is pulled in the Tech Working Group pages
should become available at wiki.social.coop. At that point the old
https://git.coop/social.coop/tech/operations repository can be [archived](https://docs.gitlab.com/ee/user/project/settings/#archive-a-project).
Show whitespace changes
Inline Side-by-side
tech-working-group/Deploying-Runko.md 0 → 100644
View page @ 849baa30
# Runko install
*This is a work-in progress, only describing the steps for a clean deployment of Mastodon v2.5.0 on our new server (at the time of writing) `runko.social.coop`. Adding the existing social.coop instance's database and customisations are not yet part of this description, nor final configurations to use S3 storage and Mailgun (they are still to be done).*
## Prerequisites
See #26, #22 and #19 (and maybe #21).
## Install docker-CE
Following guide from here:
https://docs.docker.com/install/linux/docker-ce/ubuntu/#set-up-the-repository
sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add
# You may want to alter the release name here...
echo "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable" | sudo tee /etc/apt/sources.list.d/docker.list
sudo apt update
sudo apt install docker-ce
## Install docker-compose
Oddly, there is no PPA, the suggested linux install method is to download a binary.
https://docs.docker.com/compose/install/#master-builds
sudo curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
## Checkout `sauce` and create directories
```
root@runko ~ # cd /opt/
root@runko /opt # ls
backups home lost+found root1 root2
root@runko /opt # mkdir social.coop
root@runko /opt # cd social.coop/
root@runko /opt/social.coop # git clone --depth 1 --branch rebuild root@trunk.social.coop:/var/nfs/.git sauce
root@runko /opt/social.coop # mkdir -pv $(< sauce/sibling-dirs.txt)
mkdir: created directory 'var'
mkdir: created directory 'var/lib'
mkdir: created directory 'var/lib/postgresql'
mkdir: created directory 'var/lib/postgresql/data'
mkdir: created directory 'var/lib/redis'
mkdir: created directory 'var/lib/redis/data'
mkdir: created directory 'var/www'
mkdir: created directory 'var/www/mastodon'
mkdir: created directory 'var/www/mastodon/system'
root@runko /opt/social.coop # cd sauce
root@runko /opt/social.coop/sauce # chown -R 991:991 ../var/www/mastodon/system/
```
## configure the mastodon docker container
```
root@runko /opt/social.coop/sauce # wget https://raw.githubusercontent.com/tootsuite/mastodon/v2.5.0/.env.production.sample -O .env.production
root@runko:/opt/social.coop/sauce/docker# docker-compose build
db uses an image, skipping
redis uses an image, skipping
web uses an image, skipping
streaming uses an image, skipping
sidekiq uses an image, skipping
root@runko /opt/social.coop/sauce/docker # docker-compose run --rm web bundle exec rake mastodon:setup
[images are downloaded, wizard starts]
```
To avoid errors about *"`secret_key_base` for production environment
must be a type of String"*, I find I need to work around this by
setting `SECRET_KEY_BASE` in the initial `.env.production` file to
what the Mastodon docker file uses, `precompile_placeholder`.
Now, my best judgement so far is to answer like this. The wizard does
seem very fragile, bailing out with errors on all sorts of occasions.
- **Domain name:** social.coop *(i.e. the one your server will have when the database is imported. Becuz: "Your instance is identified by its domain name. Changing it afterward will break things.")*
- **Enable single user mode?** No.
- **Are you using docker?** Yes.
- PostgreSQL:
- **host:** db *(the default)*
- **port:** 5432 *(the default)*
- **database:** mastodon-test *(don't use the default, postgres, as this cannot be deleted without clobbering your ability to log in as the postgres user later)*
- **password:** leave blank
- Redis:
- **host:** redis *(the default)*
- **port:** 6379 *(the default)*
- **password:** leave blank *(the default)*
- **Wanna store files in the cloud?** No *(easier to configure this later)*
- **Wanna send emails from localhost?** Yes *(easier to configure this later)*
- **E-mail address to send from:** Mastodon <notifications@social.coop> *(the default)*
- **Send a test email?** No *(this won't work unless you get all the details right, again eaiser to test later)*
- **Save configuration?** Yes! *(Duh. Note, it doesn't actually save your configuration, but spits it out on `stdout`. Cut and paste it into `.env.production` in the same directory as `docker-compose.yml`.)*
- **Prepare the database?** Yes *(there shouldn't be any data to lose)*
- **Compile the assets?** No *(it's slow, can be done later, and there are some in the image.)*
- **Create a an admin user?** Yes *(this avoids not being able to get confirmation mails because email out isn't configured.)*
- **Username:** I typically use `admin`
- **E-mail** `root@social.coop`, or your personal email, for now, so you can see if it works.
Given the `.env.production` this outputs, save it next to `docker-compose.yml`. (It should eventually go into our `pass` repo in the `deployment` directory).
Now, this should then allow you to run docker. This should be
safe-ish to do, since there are no users to speak of. (Details of this
is sketchy.)
docker-compose up -d
In order to make mastodon-owned files appear as a name, rather than
just the number used in the docker image, and to ensure this UID isn't
inadvertently used, we add a `mastodon` user with UID/GID 991.
groupadd -g 991 mastodon
useradd -c 'Mastodon user' -d /opt/social.coop/var/www/mastodon -g 991 -u 991 -r -s /usr/sbin/nologin mastodon
Likewise for the dockerised `postgres`:
groupadd -g 70 docker-postgres
useradd -c 'PostgreSQL user' -d /opt/social.coop/var/lib/postgresql -g 70 -u 70 -r -s /usr/sbin/nologin docker-postgres
But not for `redis`, whose UID 100 is already taken by
`systemd-network` on Ubuntu 18.04.
# Additional setup
## exim
This is for delivering mail notifications from the server and its
docker containers. Logcheck monitors the logs.
apt install exim4-daemon-light exim4 mailutils logcheck
dpkg-reconfigure exim4-config
Configured exim to be an internet server, listening on the docker
network and localhost, and relaying from docker containers (have to
check ip addr for this). The result of the exim4-config questionnaire
constructs this file:
```
root@runko /opt/social.coop/sauce # cat /etc/exim4/update-exim4.conf.conf
# /etc/exim4/update-exim4.conf.conf
#
# Edit this file and /etc/mailname by hand and execute update-exim4.conf
# yourself or use 'dpkg-reconfigure exim4-config'
#
# Please note that this is _not_ a dpkg-conffile and that automatic changes
# to this file might happen. The code handling this will honor your local
# changes, so this is usually fine, but will break local schemes that mess
# around with multiple versions of the file.
#
# update-exim4.conf uses this file to determine variable values to generate
# exim configuration macros for the configuration file.
#
# Most settings found in here do have corresponding questions in the
# Debconf configuration, but not all of them.
#
# This is a Debian specific file
dc_eximconfig_configtype='internet'
dc_other_hostnames='runko.social.coop'
dc_local_interfaces='127.0.0.1 ; 172.17.0.1 ; ::1'
dc_readhost=''
dc_relay_domains='runko.social.coop'
dc_minimaldns='false'
dc_relay_nets='172.17.0.0/16;172.18.0.0/16;172.19.0.0/16'
dc_smarthost=''
CFILEMODE='644'
dc_use_split_config='true'
dc_hide_mailname=''
dc_mailname_in_oh='true'
dc_localdelivery='maildir_home'
```
## nginx
This acts as a web-front-end for mastodon. On `trunk` it was
dockerised, but we're going with the standard Mastodon process here.
```
root@runko:/opt/social.coop/sauce# apt install nginx
```
I just then created
`/opt/social.coop/sauce/nginx/sites-available/social.coop.conf` with a
copy of the one in the [then current Mastodon production
guide](https://github.com/tootsuite/documentation/blob/45924280ca012482ec2138a549b8061d16afb2ef/Running-Mastodon/Production-guide.md),
and modified it to use the correct host and paths.
See [this commit](https://git.coop/social.coop/tech/sauce/commit/4a9f2e95f51827ea4dc425a54db6dd613da8d6ef) and the two following.
I did not yet add the changes I see in the nginx config for the docker
swarm on `trunk`. See [this branch](https://git.coop/social.coop/tech/sauce/commits/trunk-customisations) for those.
## letsencrypt ssl cert
To configure runko with an ssl cert (prior to making social.coop resolve to this host), I did:
```
root@runko:/opt/social.coop/sauce# apt install certbot
root@runko:/opt/social.coop/sauce# /etc/init.d/nginx stop
Stopping nginx (via systemctl): nginx.service.
root@runko:/opt/social.coop/sauce# certbot certonly
Saving debug log to /var/log/letsencrypt/letsencrypt.log
[...]
```
When given the choice:
```
How would you like to authenticate with the ACME CA?
-------------------------------------------------------------------------------
1: Nginx Web Server plugin - Alpha (nginx)
2: Spin up a temporary webserver (standalone)
3: Place files in webroot directory (webroot)
-------------------------------------------------------------------------------
Select the appropriate number [1-3] then [enter] (press 'c' to cancel): 2
```
I opted for 2 (my experiments with 1 failed), entered the domain
`runko.social.coop` (for now) and then the host was authenticated over
http and the certs written to
`/etc/letsencrypt/live/runko.social.coop`.
I then modified the nginx config in
`/opt/social.coop/sauce/nginx/sites-available/social.coop.conf` to
reference these.
## Testing mastodon
Given the above, starting mastodon with
root@runko /opt/social.coop/sauce/docker # /etc/init.d/nginx start
root@runko /opt/social.coop/sauce/docker # docker-cloud up -d
Nominally works, in that I see the mastodon landing page, with some
remote content, and no local content, as expected. The different
hostname would likely cause problems if used for real, but I think I
can add the correct hostname quickly when migrating the database (the
TTL for social.coop has already been set to 10 minutes IIRC - *note
this adjustment is advised in general for migrations*).

Webarchitects | Forum | Status | SSH Fingerprints