Skip to content
Snippets Groups Projects

Changes

Page history
  • New page
  • Templates
    • Update Tech Working Group authored by Akshay Mankar's avatar Akshay Mankar
    Hide whitespace changes
    Inline Side-by-side
    tech-working-group/Tech-Working-Group.md
    View page @ 7b4571d7
    # Social.Coop Tech Working Group
    This is the home of the social.coop tech group! If you are new to the group, please see the [new user guide](Getting-started-in-the-tech-group).
    This is the home of the social.coop tech group! If you are new to the group, please see the [new user guide](Getting-started-in-the-tech-group).
    - Newcomers:
    - Newcomers:
    - [Getting started](/tech-working-group/Getting-started-in-the-tech-group)
    - Procedures:
    - [How to add a social.coop email address](/tech-working-group/How-to-add-a-socialcoop-email-address) (including info on modifying other aliases like `tech.group@social.coop`)
    ......@@ -18,61 +18,62 @@ This is the home of the social.coop tech group! If you are new to the group, pl
    The tech group is responsible for operating and maintaining the following services:
    | Primary services | |
    |---|---|
    |------------------|--|
    | Social.coop mastodon instance | [admin panel](https://social.coop/admin/dashboard) |
    | wiki.social.coop public wiki | https://wiki.social.coop |
    | Supporting services | |
    |---|---|
    | Supporting services | |
    |---------------------|--|
    | registrar | [gandi](https://www.gandi.net) |
    | registrant | organization: _Xarxa integral de professionals i usuaries_ |
    | DNS / DDOS protection| [cloudflare](https://www.cloudflare.com) |
    | DNS / DDOS protection | [cloudflare](https://www.cloudflare.com) |
    | Mailgun for emails sent by mastodon | [mailgun](https://www.mailgun.com/) |
    | @social.coop email aliases | [webarch.mail](https://webarch.email/) |
    | Object store for backups and digital assets | [digital ocean spaces](https://cloud.digitalocean.com/login) |
    | Monitoring / metrics | [datadog](https://www.datadoghq.com/) |
    | Code repos | [git.coop/social.coop/tech](https://git.coop/social.coop/tech) |
    | Code repos | git.coop/social.coop/tech |
    ## Our git.coop repositories
    | Repo | purpose |
    |---|---|
    |------|---------|
    | [tech gitlab group](https://git.coop/social.coop/tech) | list of all repos |
    | [sauce](https://git.coop/social.coop/tech/sauce) | docker config and some systemd services (to be migrated)|
    | [sauce](https://git.coop/social.coop/tech/sauce) | docker config and some systemd services (to be migrated) |
    | [ansible](https://git.coop/social.coop/tech/ansible) | server configuration |
    | [pass](https://git.coop/social.coop/tech/pass) | encrypted password store for shared passwords |
    | [wiki](https://git.coop/social.coop/tech/wiki.social.coop) | code for the public metalsmith wiki |
    ## Administrative links
    - [issues](https://git.coop/social.coop/tech/operations/-/issues)
    - meetings
    - every 2 weeks on Wednesday at 19:30 UTC on even numbered [ISO weeks](https://www.epochconverter.com/weeknumbers)
    - [meeting pad](https://codi.kanthaus.online/social.coop)
    - [tech meeting minutes](https://www.loomio.org/d/UwAeiBgE/tech-meeting-minutes)
    - every 2 weeks on Monday at 19:00 UTC on odd numbered [ISO weeks](https://www.epochconverter.com/weeknumbers)
    - [meeting pad](https://codi.kanthaus.online/social.coop)
    - [tech meeting minutes](https://www.loomio.org/d/UwAeiBgE/tech-meeting-minutes)
    - communication
    - [matrix chat #socialcoop-tech:matrix.org](https://riot.im/app/#/room/#socialcoop-tech:matrix.org)
    - [tech loomio group](https://www.loomio.org/g/Mtb0FrvX/social-coop-tech-working-group)
    - [matrix chat #socialcoop-tech:matrix.org](https://riot.im/app/#/room/#socialcoop-tech:matrix.org)
    - [tech loomio group](https://www.loomio.org/g/Mtb0FrvX/social-coop-tech-working-group)
    - tech governance
    - [server access](https://www.loomio.org/d/jrbG5tue/server-access) (to vote for giving people access to server)
    - [server access](https://www.loomio.org/d/jrbG5tue/server-access) (to vote for giving people access to server)
    # Mastodon runbook
    Our fediverse instance is the raison d`etre of the social.coop coop. This is what the community signs up for and our primary responsibility. The primary points of administration are:
    Our fediverse instance is the raison d\`etre of the social.coop coop. This is what the community signs up for and our primary responsibility. The primary points of administration are:
    - the [admin panel](https://social.coop/admin/dashboard)
    - the [datadog dashboard](https://app.datadoghq.com/dash/host/640032656?from_ts=1667763442004&to_ts=1668368242004&live=true)
    - ssh cli access Access via ssh on port 2022 e.g. `ssh user@runko.social.coop -p 2022`.
    - ssh cli access Access via ssh on port 2022 e.g. `ssh user@runko.social.coop -p 2022`.
    If you need server access please see [server access](https://www.loomio.org/d/jrbG5tue/server-access) and [How to grant shell access](https://git.coop/social.coop/tech/operations/-/wikis/how-to-get-server-shell-access)
    Mastodon is running under [Docker-compose](https://docs.docker.com/compose/).
    Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. We are using Systemd as the init system for docker-compose.
    Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. We are using Systemd as the init system for docker-compose.
    ## systemd services
    | service | purpose |
    |---|---|
    |---------|---------|
    | social.coop-mastodon | a service to control the mastodon installation via docker-compose |
    | social.coop-remove-media | runs the media cleanup command to remove remote media >7 days old via a .timer |
    | certbot | runs the renewals via .timer |
    ......@@ -80,7 +81,7 @@ Compose is a tool for defining and running multi-container Docker applications.
    ## logs
    | command | purpose |
    |---|---|
    |---------|---------|
    | systemctl list-timers | lists timers! |
    | journalctl -f | tail ALL system logs |
    | docker-compose logs -f web | view and tail web logs (when in `/opt/social.coop/sauce/docker/`) |
    ......@@ -92,8 +93,9 @@ Compose is a tool for defining and running multi-container Docker applications.
    ## Service management
    All of these commands must be run on runko.social.coop in the `/opt/social.coop/sauce/docker/` directory.
    | command | purpose |
    |---|---|
    |---------|---------|
    | `docker-compose ps` | List all Docker containers |
    | `docker-compose stop redis` | Stop a service |
    | `docker-compose start redis` | Start a service |
    ......@@ -118,27 +120,19 @@ Location of Postgres database files: `/opt/social.coop/var/lib/postgresql/data/`
    - check upgrade notes
    - check whether there are DB migrations
    - make backup? `systemctl start pg-dump-to-s3.service`
    - takes 15 mins or so?
    - takes 15 mins or so?
    - separate command to see backup progress
    - make merge request on git.coop sauce repo to bump version in a couple of
    places in docker-compose.yaml
    - `git diff v3.1.2..v3.1.3 -- docker-compose.yml` in mastodon repo after pulling
    to check whether there were any changes we should consider mirroring to our
    docker-compose file
    - make merge request on git.coop sauce repo to bump version in a couple of places in docker-compose.yaml
    - `git diff v3.1.2..v3.1.3 -- docker-compose.yml` in mastodon repo after pulling to check whether there were any changes we should consider mirroring to our docker-compose file
    - could be cool to make these merge requests in advance
    - write a toot announcing upgrade and boost on admin account
    - touch file on server to activate maintenance mode
    - actually do the upgrade
    - migration command creates a fresh web container and runs the migration
    command and then deletes that new container
    - migration command creates a fresh web container and runs the migration command and then deletes that new container
    - turn maintenance mode off
    - we copy static assets outside of the container so they can be served by nginx
    - there's a command for this which moves stuff into a temporary dir in nginx
    and pulls assets out of docker container into that folder in docker
    container
    - ssh forwarding is nice, then with `sudo -E -s` you have ssh access to stuff
    you do from host machine(?)
    - there's a command for this which moves stuff into a temporary dir in nginx and pulls assets out of docker container into that folder in docker container
    - ssh forwarding is nice, then with `sudo -E -s` you have ssh access to stuff you do from host machine(?)
    ## Hardware
    ......@@ -166,7 +160,6 @@ sdb 8:16 0 223.6G 0 disk
    └─vg0-opt 253:2 0 396.1G 0 lvm /opt
    ```
    ```
    LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
    opt vg0 -wi-ao---- 396.13g
    ......@@ -176,10 +169,9 @@ sdb 8:16 0 223.6G 0 disk
    `opt` is mounted at `/opt`.
    # wiki.social.coop
    [wiki.social.coop](https://wiki.social.coop) has two main purposes:
    wiki.social.coop has two main purposes:
    1. a public-facing site with information about social.coop
    2. the registration system for new users
    ......@@ -189,6 +181,7 @@ The code repo for the project is [tech/wiki.social.coop](https://git.coop/social
    It's configured/deployed via ansible using the [wiki.social.coop role](https://git.coop/social.coop/tech/ansible/-/tree/master/roles/wiki.social.coop) and the `wiki` tag, so `ansible-playbook server.playbook.yml --tags wiki` will set it up.
    The configuration secrets are stored in the [pass repo](https://git.coop/social.coop/tech/pass) at:
    ```
    deployment/wiki/gitlab_token
    deployment/wiki/gitlab_username
    ......

    Webarchitects | Forum | Status | SSH Fingerprints