Confluence 6.0 introduced Synchrony (allowing collaborative editing) which cannot accept AJP connections. The preferred configuration is Using Apache with mod_proxy.

Unfortunately, I believe there is a mistake in the Confluence instructions... so after posting my questions here are my notes to figure this all out.

Version Log

These instructions have proven to work with,

Ubuntu

Java

Confluence

Apache

Verifier

16.04 LTS in OpenVZ

Kernel 2.6.32-042stab139.1

JRE 8u221 64-bit

6.15.7

2.4.18

Tin

Setup Apache Reverse Proxy

Enable the following the proxy_http module which will also enable the main module proxy,

sudo a2enmod proxy_http

Confluence 6.0 or later uses Synchrony (required for collaborative editing) and needs these modules which are only available for Apache 2.4 or later,

sudo a2enmod proxy_wstunnel
sudo a2enmod rewrite

Make sure your sites works as expected with your virtual host.

Setup Confluence 5.x to 6.x Migration

There are other ways of doing this, but this is what I did that worked and is tested so far.

Setup base Confluence without worrying about proxy. There's nothin new here from the 5.x instructions.

Disabled the firewall,

...

Load URL... www.krypton.com:8090

Select "Production Installation"
Select apps (in my case Confluence Questions)
Enter your licenses
My own database
Enter in the database information
Upload and restore your files to /opt/confluence-data/restore - make sure to copy the file in as serveradmin user for proper permissions
Restore From Backup (if you forgot to upload the file, you can still do it at this step and click refresh on our browser and ensure recreate index is clicked

Login with your admin account. Confirm using Chrome's Developer Tools that there are no errors being thrown.

Shutdown Confluence.

Configure the server.xml file.

Log in as admin using through the apache web server url, www.krypton.com/wiki

You'll notice a dialog box,

Your URL doesn't match.

Click the link in the dialogue box to "Update base URL"

It will log into admin. Change,

http://www.krypton.com:8090

to

http://www.krypton.com/wiki

Click save.

Log in as admin using through the apache web server url, www.krypton.com/wiki

You'll notice a dialog box,

Your URL doesn't match.

Click the link in the dialogue box to "Update base URL"

It will log into admin. Change,

http://www.krypton.com:8090

to

http://www.krypton.com/wiki

Click save.

Key Test Cases to Ensure Synchrony works,

Using Developer Tools
Trying out Collaborative Editing

Trouble Shooting First Setup

Just could not properly connect from the web server and getting strange errors.

Analysis 1

I believe I've found the error. It probably has to do with the search engine. On the test server's application server - with more memory and updated OS and newer Java - making this batch.js request results in a response. However, on the restricted environment it does not,

wget http://localhost:8090/wiki/s/ca6b4214dc4957e5b90edfd999cd552a-CDN/en_US/8100/e6b42acaec191a9bce4ab234dae0b1b08af15496/e00352cb0013a1867dc5de3984f0fae4/_/download/contextbatch/js/confluence-search-ui-plugin-main,-_super/batch.js?locale=en-US

Let's start with Java,

Not working server = java version "1.8.0_74" 32-bit
Working = java version "1.8.0_221" 64-bit

Now with matching Java version at least, but not 64-bit.

I noticed that "synchrony version" does not show up on startup of the system with the issue,

04-Aug-2019 21:50:06.328 INFO [Catalina-utility-2] org.springframework.web.context.support.AnnotationConfigWebApplicationContext.loadBeanDefinitions Registering annotated classes: [class com.atlassian.synchrony.proxy.websocket.WebSocketConfig,class com.atlassian.synchrony.proxy.web.SynchronyWebMvcConfig]
2019-08-04 21:50:06,648 INFO [Catalina-utility-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 6.15.7 [build 8100 based on commit hash e6b42acaec191a9bce4ab234dae0b1b08af15496] - synchrony version 2.1.0-release-confluence_6.15-32f7299a
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation

Analysis 2

Now with matching Java version at least, but not 64-bit.

I noticed that "synchrony version" does not show up on startup of the system with the issue,

04-Aug-2019 21:50:06.328 INFO [Catalina-utility-2] org.springframework.web.context.support.AnnotationConfigWebApplicationContext.loadBeanDefinitions Registering annotated classes: [class com.atlassian.synchrony.proxy.websocket.WebSocketConfig,class com.atlassian.synchrony.proxy.web.SynchronyWebMvcConfig]
2019-08-04 21:50:06,648 INFO [Catalina-utility-1] [com.atlassian.confluence.lifecycle] contextInitialized Starting Confluence 6.15.7 [build 8100 based on commit hash e6b42acaec191a9bce4ab234dae0b1b08af15496] - synchrony version 2.1.0-release-confluence_6.15-32f7299a
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation

Ok same Java version including 64-bit...

Solution!

And that was it!

Make sure to adjust Confluence's context path through the admin console Server Base URL, in my case to www.breakitdown/wiki Here is my current working files,

/opt/confluence/conf/server.xml

<Server port="8000" shutdown="SHUTDOWN" debug="0">
    <Service name="Tomcat-Standalone">
        <!--
         ==============================================================================================================
         DEFAULT - Direct connector with no proxy, for unproxied HTTP access to Confluence.

         If using a http/https proxy, comment out this connector.
         ==============================================================================================================
        -->
<!--
        <Connector port="8090" connectionTimeout="20000" redirectPort="8443"
                   maxThreads="48" minSpareThreads="10"
                   enableLookups="false" acceptCount="10" debug="0" URIEncoding="UTF-8"
                   protocol="org.apache.coyote.http11.Http11NioProtocol"/>
-->
        <!--
         ==============================================================================================================
         HTTP - Proxying Confluence via Apache or Nginx over HTTP

         If you're proxying traffic to Confluence over HTTP, uncomment the connector below and comment out the others.
         Make sure you provide the right information for proxyName and proxyPort.

         For more information see:
            Apache - https://confluence.atlassian.com/x/4xQLM
            nginx  - https://confluence.atlassian.com/x/TgSvEg

         ==============================================================================================================
        -->

        <Connector port="8090" connectionTimeout="20000" redirectPort="8443"
                   maxThreads="48" minSpareThreads="10"
                   enableLookups="false" acceptCount="10" debug="0" URIEncoding="UTF-8"
                   protocol="org.apache.coyote.http11.Http11NioProtocol"
                   scheme="http" proxyName="www.breakitdown.ca" proxyPort="80"/>

        <!--
         ==============================================================================================================
         HTTPS - Direct connector with no proxy, for unproxied HTTPS access to Confluence.

         For more info see https://confluence.atlassian.com/x/s3UC
         ==============================================================================================================
        -->

        <!--
        <Connector port="8443" maxHttpHeaderSize="8192"
                   maxThreads="150" minSpareThreads="25"
                   protocol="org.apache.coyote.http11.Http11Nio2Protocol"
                   enableLookups="false" disableUploadTimeout="true"
                   acceptCount="100" scheme="https" secure="true"
                   clientAuth="false" sslProtocol="TLSv1.2" sslEnabledProtocols="TLSv1.2" SSLEnabled="true"
                   URIEncoding="UTF-8" keystorePass="<MY_CERTIFICATE_PASSWORD>"/>
        -->

        <!--
         ==============================================================================================================
         HTTPS - Proxying Confluence via Apache or Nginx over HTTPS

         If you're proxying traffic to Confluence over HTTPS, uncomment the connector below and comment out the others.
         Make sure you provide the right information for proxyName and proxyPort.

         For more information see:
            Apache - https://confluence.atlassian.com/x/PTT3MQ
            nginx  - https://confluence.atlassian.com/x/cNIvMw
         ==============================================================================================================
        -->

        <!--
        <Connector port="8090" connectionTimeout="20000" redirectPort="8443"
                   maxThreads="48" minSpareThreads="10"
                   enableLookups="false" acceptCount="10" debug="0" URIEncoding="UTF-8"
                   protocol="org.apache.coyote.http11.Http11NioProtocol"
                   scheme="https" secure="true" proxyName="<subdomain>.<domain>.com" proxyPort="443"/>
        -->

        <Engine name="Standalone" defaultHost="localhost" debug="0">
            <Host name="localhost" debug="0" appBase="webapps" unpackWARs="true" autoDeploy="false" startStopThreads="4">
                <Context path="/wiki" docBase="../confluence" debug="0" reloadable="false" useHttpOnly="true">
                    <!-- Logging configuration for Confluence is specified in confluence/WEB-INF/classes/log4j.properties -->
                    <Manager pathname=""/>
                    <Valve className="org.apache.catalina.valves.StuckThreadDetectionValve" threshold="60"/>
                </Context>

                <Context path="${confluence.context.path}/synchrony-proxy" docBase="../synchrony-proxy" debug="0"
                         reloadable="false" useHttpOnly="true">
                    <Valve className="org.apache.catalina.valves.StuckThreadDetectionValve" threshold="60"/>
                </Context>
            </Host>
        </Engine>
    </Service>
</Server>

You can test this still by directly hitting the 8090 port.

And http configuration where my hosts file points to the application server,

<VirtualHost *:80>
        ServerAdmin webmaster@t01app

        ServerName www.breakitdown.ca
        ServerAlias breakitdown.ca

        # Enforce www in front of url.
        RewriteEngine On
        RewriteCond %{HTTP_HOST} !^www\.breakitdown\.ca$ [NC]
        RewriteRule (.*) http://www.breakitdown.ca$1 [R,L]

        DocumentRoot /opt/www.breakitdown.ca/www

        <Directory /opt/www.breakitdown.ca/www/>
                Options Indexes FollowSymLinks
                AllowOverride None
                Require all granted
                DirectorySlash Off
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/www.breakitdown.ca.error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        CustomLog ${APACHE_LOG_DIR}/www.breakitdown.ca.access.log combined


        ProxyRequests Off
        ProxyPreserveHost On

        ProxyPass /synchrony http://t01app:8091/synchrony
        <Location /synchrony>
                Require all granted
                RewriteEngine on
                RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
                RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
                RewriteRule .* ws://t01app:8091%{REQUEST_URI} [P]
        </Location>

        # Can probably remove this as it looks like a fallback for "/synchrony". Will do once I've had a chance to retest the system.
        ProxyPass /synchrony-proxy http://t01app:8091/synchrony-proxy
        <Location /synchrony-proxy>
                Require all granted
                RewriteEngine on
                RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
                RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
                RewriteRule .* ws://t01app:8091%{REQUEST_URI} [P]
        </Location>

        # I noticed errors in the browser logs without this. Looks like Atlassian coders may have some hard coding of using "/s" for synchrony.  
        ProxyPass /s http://t01app:8091/synchrony
        <Location /s>
                Require all granted
                RewriteEngine on
                RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
                RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
                RewriteRule .* ws://t01app:8091%{REQUEST_URI} [P]
        </Location>

        <Proxy *>
                Require all granted
        </Proxy>
        ProxyPass /wiki http://t01app:8090/wiki
        ProxyPassReverse /wiki http://t01app:809/wiki

        # Perform 301 permanent redirect to Board of Trustees Home
        #Redirect permanent /board/ http://www.breakitdown.ca/wiki/display/board/Home
        #Redirect permanent /board http://www.breakitdown.ca/wiki/display/board/Home

</VirtualHost>