diff --git a/README.md b/README.md index 100782ccaa1f7b0aac43a6280ed7daed60fba288..924ab0c2179d7d54e605f371bbc1c8eed3e37da0 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,24 @@ These Playbooks are designed to be used on Debian Stretch virtual servers. ## Discourse -Login to the virtual server console, install `python`, enable root ssh access -using keys by adding your keys to `/root/.ssh/authorized_keys`, edit +Ansible Playbooks to install +[Docker](https://store.docker.com/editions/community/docker-ce-server-debian) +and [Discourse](https://github.com/discourse/discourse_docker) on a Debian +Stretch virtual server and to configure the virtual server to use Postfix for +incoming and outgoing emails (there is also a not-quite-working and, for now, +abandoned [exim branch](https://git.coop/cotech/ansible/tree/exim)). + +The email setup is based on the [mail-reciever Docker +container](https://github.com/discourse/mail-receiver) plus [this pull +request](https://github.com/discourse/mail-receiver/pull/2) (which is now +merged) and the [Postfix notes for using the host for outgoing +email](https://meta.discourse.org/t/emails-with-local-smtp/23645/28), with an +additional [Ruby +script](https://git.coop/cotech/ansible/blob/master/roles/email/files/discourse-smtp-rcpt-acl). + +Before running these Playbooks, create a virtual server, runnng Debian Stretch +then login to the virtual server's console, install `python`, enable root ssh +access using keys by adding your keys to `/root/.ssh/authorized_keys`, edit `/etc/sshd/sshd_config` to set `PermitRootLogin prohibit-password`, run `service ssh restart` and then run the first Playbook: @@ -40,24 +56,17 @@ then the `iptables` and `munin-node` roles will, as a minimum, need editing and might be best omitted. Also note that these Playbooks are based on using `mx.webarch.net` for incoming email -- this is an anti-spam gateway, if this wasn't used then SpamAssassin should probably be added to the mix. - -The email setup was originally based on the -[mail-reciever](https://github.com/discourse/mail-receiver) Docker container -plus the [outstanding pull -request](https://github.com/discourse/mail-receiver/pull/2) and the [Postfix -notes](https://meta.discourse.org/t/emails-with-local-smtp/23645/28) for using -the host for outgoing email, but then we switched it over to use Exim. ### CoTech Community Discourse Settings Initial settings used for `community.coops.tech` when it was created: * title: Cooperative Technologists Community -* site description: The intersection of co-operation and technology, the CoTech community forum. -* contact email: community@coops.tech -* contact url: https://www.coops.tech/ -* notification email: discourse@community.coops.tech -* site contact username: system +* site description: The intersection of co-operation and digtal technology, the CoTech community forum. +* contact email: `community@coops.tech` +* contact url: `https://www.coops.tech/` +* notification email: `discourse@community.coops.tech` +* site contact username: `system` * logo url: https://wiki.coops.tech/wiki/File:Cotech-blue.png * logo small url: https://wiki.coops.tech/wiki/File:Cotech-blue-text.png * company short name: CoTech @@ -70,7 +79,7 @@ On the Email settings admin page: * reply by email enabled * reply by email address: `discourse+%{reply_key}@community.coops.tech` * manual polling enabled -* email prefix: cotech-community +* email prefix: `cotech-community` * email site title: CoTech Community On the Security page: @@ -87,14 +96,6 @@ On the User Preferences page: The first post text: -Welcome to the **Cooperative Technologists Community**, we are a network of technology focused cooperatives, [CoTech](https://www.coops.tech/), who are *"building a tech industry that's better for its workers and customers through co-operation, democracy and worker ownership."* This is our open community discussion forum, you don't have to be a member of a coop to join this community but you do need to support [the cooperative values and principles](http://ica.coop/en/whats-co-op/co-operative-identity-values-principles) and have an interest in technology, you can find out more [about us](https://www.coops.tech/about), read [our manifesto](https://www.coops.tech/manifesto), see who we are and who we have worked for and watch [a video made at our first gathering](https://vimeo.com/196080655) on [www.coops.tech](https://www.coops.tech/). We also have [a wiki](https://wiki.coops.tech/) and a decision making group on [Loomio](https://www.loomio.org/g/oVwtKDOn/digital-co-ops), [Slack channels](https://tech-coops.slack.com/) and (for now, we might close it and use Discourse) a public [email list](https://www.email-lists.org/mailman/listinfo/tech-coops). +Welcome to the **Cooperative Technologists Community**, we are a network of technology focused digital cooperatives, [CoTech](https://www.coops.tech/), who are *"building a tech industry that's better for its workers and customers through co-operation, democracy and worker ownership."* This is our open community discussion forum, you don't have to be a member of a coop to join this community but you do need to support [the cooperative values and principles](http://ica.coop/en/whats-co-op/co-operative-identity-values-principles) and have an interest in technology, you can find out more [about us](https://www.coops.tech/about), read [our manifesto](https://www.coops.tech/manifesto), see who we are and who we have worked for and watch [a video made at our first gathering](https://vimeo.com/196080655) on [www.coops.tech](https://www.coops.tech/). We also have [a wiki](https://wiki.coops.tech/) and a decision making group on [Loomio](https://www.loomio.org/g/oVwtKDOn/digital-co-ops), [Slack channels](https://tech-coops.slack.com/) and (for now, we might close it and use Discourse) a public [email list](https://www.email-lists.org/mailman/listinfo/tech-coops). *Please read [our community guidelines](https://community.coops.tech/guidelines) before signing up for an account here.* - - - - - - - - diff --git a/roles/api/tasks/main.yml b/roles/api/tasks/main.yml index 10e087446f3e1bcf3b8d8f886b84f2b4deb86587..8c3c1e9019856602da7035f03da932ae0dfdf571 100644 --- a/roles/api/tasks/main.yml +++ b/roles/api/tasks/main.yml @@ -1,7 +1,7 @@ --- -- name: Stat "/etc/exim4/mail-receiver-environment.json" +- name: Stat "/etc/postfix/mail-receiver-environment.json" stat: - path: "/etc/exim4/mail-receiver-environment.json" + path: "/etc/postfix/mail-receiver-environment.json" register: mail_receiver_environment - block: @@ -9,9 +9,9 @@ - name: Discourse scripts environmental variables file in place template: src: templates/mail-receiver-environment.json.j2 - dest: /etc/exim4/mail-receiver-environment.json - mode: 0640 - group: Debian-exim + dest: /etc/postfix/mail-receiver-environment.json + mode: 0644 + group: root owner: root when: mail_receiver_environment.stat.exists == False diff --git a/roles/discourse/tasks/main.yml b/roles/discourse/tasks/main.yml index 25b97e4ca4648b4f93460315112478a1c31fc6d3..e66a0054c3d6278ca3ff09eda207cd7fad59eea9 100644 --- a/roles/discourse/tasks/main.yml +++ b/roles/discourse/tasks/main.yml @@ -1,13 +1,114 @@ --- +- name: Group for Discourse present + group: + name: discourse + system: yes + state: present + gid: 1000 + +- name: User for Discourse present + user: + name: discourse + system: yes + state: present + shell: /bin/bash + home: /home/discourse + createhome: true + groups: discourse,docker + uid: 1000 + +- name: Stat /var/discourse/lost+found + stat: + path: "/var/discourse/lost+found" + register: var_discourse_partition + +- block: + + - name: Delete lost+found directory if /var/discourse is a partition + file: + dest: /var/discourse/lost+found + state: absent + + when: var_discourse_partition.stat.exists == True + - name: Directory for Discourse present file: dest: /var/discourse state: directory + owner: discourse + group: discourse + +- name: ssl-cert group present for UID mappings + group: + name: ssl-cert + system: yes + state: present + gid: 111 + +- name: postgres group present for UID mappings + group: + name: postgres + system: yes + state: present + gid: 112 + +- name: postgres user persent for GID mappings + user: + name: postgres + system: yes + group: postgres + createhome: false + shell: /bin/false + uid: 107 + +- name: haproxy group present for UID mappings + group: + name: haproxy + system: yes + state: present + gid: 113 + +- name: haproxy user persent for GID mappings + user: + name: haproxy + system: yes + group: haproxy + createhome: false + shell: /bin/false + uid: 108 + +- name: redis group present for UID mappings + group: + name: redis + system: yes + state: present + gid: 114 + +- name: redis user persent for GID mappings + user: + name: redis + system: yes + group: redis + createhome: false + shell: /bin/false + uid: 110 - name: Discourse checked out git: repo: https://github.com/discourse/discourse_docker.git dest: /var/discourse + become: yes + become_user: 'discourse' + +- block: + + - name: Create lost+found directory + command: mklost+found + args: + chdir: /var/discourse + creates: /var/discourse/lost+found + + when: var_discourse_partition.stat.exists == True - name: Count how much swap is available shell: "free -g --si | awk '/^Swap:/{print $2}'" @@ -79,6 +180,10 @@ template: src: templates/standalone.yml.j2 dest: /var/discourse/containers/app.yml + become: yes + become_user: discourse - name: Rebuild Discourse app command: /var/discourse/launcher rebuild app + become: yes + become_user: discourse diff --git a/roles/docker/tasks/main.yml b/roles/docker/tasks/main.yml index cef91ff7c11b9f24561d2e54ce52f12fcf64a4f7..f5f66648de231eaf6c80b50c3a575bb2ccee0ea6 100644 --- a/roles/docker/tasks/main.yml +++ b/roles/docker/tasks/main.yml @@ -6,7 +6,10 @@ update_cache: yes with_items: - apt-transport-https + - ca-certificates + - curl - git + - software-properties-common - name: Docker GPG key present apt_key: @@ -24,3 +27,8 @@ name: docker-ce state: present update_cache: yes + +- name: Docker started + service: + name: docker + state: started diff --git a/roles/email/files/00_local b/roles/email/files/00_local deleted file mode 100644 index 97ea365ba53966c87d5f91883189355725185250..0000000000000000000000000000000000000000 --- a/roles/email/files/00_local +++ /dev/null @@ -1 +0,0 @@ -CHECK_RCPT_LOCAL_ACL_FILE = CONFDIR/check_rcpt_local_acl diff --git a/roles/email/files/30_exim4-config_discourse b/roles/email/files/30_exim4-config_discourse deleted file mode 100644 index 1c1635b17b8d4597fddec358e4ee1fdb14b809c4..0000000000000000000000000000000000000000 --- a/roles/email/files/30_exim4-config_discourse +++ /dev/null @@ -1,4 +0,0 @@ -discourse_transport: - driver = pipe - command = /usr/local/bin/receive-mail ${local_part}@${domain} - diff --git a/roles/email/files/450_exim4-config_discourse b/roles/email/files/450_exim4-config_discourse deleted file mode 100644 index 50d92b4b3c86a27988f2af1375eb97757eb900ea..0000000000000000000000000000000000000000 --- a/roles/email/files/450_exim4-config_discourse +++ /dev/null @@ -1,4 +0,0 @@ -discourse_router: - driver = accept - transport = discourse_transport - diff --git a/roles/email/files/check_rcpt_local_acl b/roles/email/files/check_rcpt_local_acl deleted file mode 100644 index 024108e5cbad44ecb3121f4be66070bb7467a2f1..0000000000000000000000000000000000000000 --- a/roles/email/files/check_rcpt_local_acl +++ /dev/null @@ -1,14 +0,0 @@ -# Local rcpt check - deny - message = No such discourse list - log_message = No such discourse list - !acl = acl_local_deny_exceptions - condition = ${run{/usr/local/bin/discourse-smtp-rcpt-acl $sender_address $local_part@$domain}{no}{${if eq {$runrc}{2}{yes}{no}}}} - - defer - message = Temporary error checking discourse - !acl = acl_local_deny_exceptions - condition = ${if eq {$runrc}{1}{yes}{no}} - - - diff --git a/roles/email/files/discourse-smtp-fast-rejection b/roles/email/files/discourse-smtp-fast-rejection index 887bc38c8d94583f4c65732b13f3eae7c598c878..e3cf6515302b7fa432457f3887079a9af85530ac 100644 --- a/roles/email/files/discourse-smtp-fast-rejection +++ b/roles/email/files/discourse-smtp-fast-rejection @@ -6,7 +6,7 @@ require 'uri' require 'cgi' require 'net/http' -ENV_FILE = "/etc/exim4/mail-receiver-environment.json" +ENV_FILE = "/etc/postfix/mail-receiver-environment.json" def logger @logger ||= Syslog.open("smtp-reject", Syslog::LOG_PID, Syslog::LOG_MAIL) @@ -72,7 +72,11 @@ def maybe_reject_email(from, to, env) endpoint = "#{env['DISCOURSE_BASE_URL']}/admin/email/smtp_should_reject.json" key = env["DISCOURSE_API_KEY"] username = env["DISCOURSE_API_USERNAME"] - + # just maker sure we have something in the from field + # so we can test for addresses remotely + if from == '' + from = 'test@example.org' + end uri = URI.parse(endpoint) fromarg = CGI::escape(from) toarg = CGI::escape(to) diff --git a/roles/email/files/discourse-smtp-rcpt-acl b/roles/email/files/discourse-smtp-rcpt-acl index a1d5255d9689088b092be957879273a717e57e06..f4df5822e8c25d3bbf29a35a0a625e1b58d83ab2 100644 --- a/roles/email/files/discourse-smtp-rcpt-acl +++ b/roles/email/files/discourse-smtp-rcpt-acl @@ -10,7 +10,7 @@ require 'net/http' # Returns 1 for defer # Returns 2 for reject -ENV_FILE = "/etc/exim4/mail-receiver-environment.json" +ENV_FILE = "/etc/postfix/mail-receiver-environment.json" def logger @logger ||= Syslog.open("smtp-reject", Syslog::LOG_PID, Syslog::LOG_MAIL) diff --git a/roles/email/files/receive-mail b/roles/email/files/receive-mail index 2b5183975dbf8ddbc1750efc5c3460df50690ca3..5164af9baa8c22a5b909bcd06065856212c065b4 100644 --- a/roles/email/files/receive-mail +++ b/roles/email/files/receive-mail @@ -1,6 +1,6 @@ #!/usr/bin/env ruby -ENV_FILE = "/etc/exim4/mail-receiver-environment.json" +ENV_FILE = "/etc/postfix/mail-receiver-environment.json" EX_TEMPFAIL = 75 EX_SUCCESS = 0 diff --git a/roles/email/tasks/main.yml b/roles/email/tasks/main.yml index e0b85773ce06c115f92559d8e97614b9fefca39c..f17a10352c91a05a0ca1099597049c7016df4c8b 100644 --- a/roles/email/tasks/main.yml +++ b/roles/email/tasks/main.yml @@ -29,7 +29,26 @@ dest: /usr/local/bin/discourse-smtp-rcpt-acl mode: 0755 -- name: Exim and related email packages installed +- name: debconf-utils installed for Ansible + apt: + name: debconf-utils + state: present + +- name: Debconf Postfix hostname set + debconf: + name: postifx + question: "postfix/mailname" + value: "{{ hostname }}" + vtype: string + +- name: Debconf Postfix set to be a internet server + debconf: + name: postfix + question: "postfix/main_mailer_type" + value: "Internet Site" + vtype: string + +- name: Postfix and related email packages installed apt: pkg: "{{ item }}" state: latest @@ -38,31 +57,20 @@ - curl - debian-archive-keyring - dnsutils - - exim4-daemon-light - mailutils - mutt + - postfix - pwgen - whois -- name: Exim check_rcpt_local_acl in place - copy: - src: files/check_rcpt_local_acl - dest: /etc/exim4/check_rcpt_local_acl +- name: Postfix smtpd_relay_restrictions set + command: postconf -e "smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination" -- name: Exim 00_local in place - copy: - src: files/00_local - dest: /etc/exim4/conf.d/main/00_local +- name: Postfix set not to use /etc/aliases + command: postconf -e "alias_maps = " -- name: Exim 450_exim4-config_discourse in place - copy: - src: files/450_exim4-config_discourse - dest: /etc/exim4/conf.d/router/450_exim4-config_discourse - -- name: Exim 30_exim4-config_discourse in place - copy: - src: files/30_exim4-config_discourse - dest: /etc/exim4/conf.d/transport/30_exim4-config_discourse +- name: Postfix mydestination set to localhost + command: postconf -e "mydestination = localhost" - name: Get the app container IP address command: "docker inspect --format '{''{ .NetworkSettings.IPAddress }''}' app" @@ -71,16 +79,70 @@ - debug: msg: "The Discourse app Docker container has the IP address {{ app_ip_address.stdout }}" -- name: Exim config in place +- name: Postfix my networks set to include {{ app_ip_address.stdout }} + command: postconf -e "mynetworks = 127.0.0.0/8, {{ app_ip_address.stdout }}" + +- name: Postfix relay domains set to {{ hostname }} + command: postconf -e "relay_domains = {{ hostname }}" + +- name: Postfix smtpd_recipient_restrictions set + command: postconf -e "smtpd_recipient_restrictions = permit_mynetworks, check_policy_service unix:private/policy" + +- name: Postfix opportunistic TLS enabled + command: postconf -e "smtp_tls_security_level = may" + +- name: Postfix set to use sub-addresing + command: postconf -e "recipient_delimiter = +" + +- name: Postfix disable UTF-8 SMTP input + command: postconf -e "smtputf8_enable=no" + +- name: Postfix Time Zone and Lang set + command: postconf -e "export_environment='TZ LANG'" + +- name: Postfix set for ipv4 only + command: postconf -e "inet_protocols = ipv4" + +- name: Postfix set to use /usr/local/bin/receive-mail + command: postconf -M -e "discourse/unix=discourse unix - n n - - pipe user=nobody:nogroup argv=/usr/local/bin/receive-mail ${recipient}" + +- name: Postfix transport in place template: - src: templates/update-exim4.j2 - dest: /etc/exim4/update-exim4.conf.conf + src: templates/transport.j2 + dest: /etc/postfix/transport + mode: 0644 + +- name: Postfix Transport Maps file set + command: postconf -e "transport_maps=hash:/etc/postfix/transport" + +- name: Postmap run with Transport Maps file + command: postmap /etc/postfix/transport -- name: Exim reconfigured - command: update-exim4.conf +- name: Postfix set to reject incorrect email addresses + command: postconf -M -e "policy/unix=policy unix - n n - - spawn user=nobody argv=/usr/local/bin/discourse-smtp-fast-rejection" + +- name: Stat "/var/discourse/shared/standalone/letsencrypt/{{ hostname }}/{{ hostname }}.cer" + stat: + path: "/var/discourse/shared/standalone/letsencrypt/{{ hostname }}/{{ hostname }}.cer" + register: le_cert + +- block: + + - name: Postfix configured to use Let's Encrypt RSA cert for incoming email + command: postconf -e "smtpd_tls_cert_file = /var/discourse/shared/standalone/letsencrypt/{{ hostname }}/{{ hostname }}.cer" + + - name: Postfix configured to use Let's Encrypt RSA key for incoming email + command: postconf -e "smtpd_tls_key_file = /var/discourse/shared/standalone/letsencrypt/{{ hostname }}/{{ hostname }}.key" + + when: le_cert.stat.exists == True + +- name: Postfix stopped + command: postfix stop + +- name: Postfix started + command: postfix start - name: Root .forward in place template: src: templates/forward.j2 dest: /root/.forward - diff --git a/roles/email/templates/transport.j2 b/roles/email/templates/transport.j2 new file mode 100644 index 0000000000000000000000000000000000000000..e4f9e67e393326ec9f5f7669567c827482b1aa13 --- /dev/null +++ b/roles/email/templates/transport.j2 @@ -0,0 +1 @@ +{{ hostname }} discourse: diff --git a/roles/email/templates/update-exim4.j2 b/roles/email/templates/update-exim4.j2 deleted file mode 100644 index 883a8ab9411f01f5772213dc2ce024967393341f..0000000000000000000000000000000000000000 --- a/roles/email/templates/update-exim4.j2 +++ /dev/null @@ -1,32 +0,0 @@ -# /etc/exim4/update-exim4.conf.conf -# -# Edit this file and /etc/mailname by hand and execute update-exim4.conf -# yourself or use 'dpkg-reconfigure exim4-config' -# -# Please note that this is _not_ a dpkg-conffile and that automatic changes -# to this file might happen. The code handling this will honor your local -# changes, so this is usually fine, but will break local schemes that mess -# around with multiple versions of the file. -# -# update-exim4.conf uses this file to determine variable values to generate -# exim configuration macros for the configuration file. -# -# Most settings found in here do have corresponding questions in the -# Debconf configuration, but not all of them. -# -# This is a Debian specific file - -dc_eximconfig_configtype='internet' -dc_other_hostnames='{{ hostname }}' -dc_local_interfaces='' -dc_readhost='' -dc_relay_domains='' -dc_minimaldns='false' -dc_relay_nets='{{ app_ip_address.stdout }}/32' -dc_smarthost='' -CFILEMODE='644' -dc_use_split_config='true' -dc_hide_mailname='' -dc_mailname_in_oh='true' -dc_localdelivery='mail_spool' -