proxies: more changes for the sporadic 502 issue ( 12913 )

I got claude to dig into the things we tried to fix this issue and look
for things we missed. It found a few things:

1. In commit 35a1b3223b Victor Koycheff added some proxyopts for
   keepalive, etc. However, unfortunately, while the variable was set
   the template wasn't looking at that, so they were never actually
   set in the website. ;( So, we fix that by passing the variable in the
   right template here.

2. kojihub didn't have a keepalive set on the backend. (but this is
   likely cosmetic since the problem is at the proxy layer). We fix
   this by adding one anyhow.

3. The pass thru anubis didn't have keepalive set right, so we do
   that in the template. This may fix other 502 issues with other
   applications also.

Calude used 57,508 tokens looking at all this. ;)

Assisted-By: claude

Signed-off-by: Kevin Fenzi <kevin@scrye.com>

roles/httpd/reverseproxy/templates/reversepassproxy.conf-nonopenshift-rdu3

@@ -56,7 +56,7 @@ RewriteRule .* "balancer://{{ balancer_name }}-websocket%{REQUEST_URI}" [P]
     {% endif %}
   {% endfor %}
 </Proxy>
-ProxyPass {{ localpath }} "balancer://{{balancer_name}}{{remotepath}}"
+ProxyPass {{ localpath }} "balancer://{{balancer_name}}{{remotepath}}" {{ proxyopts|default('') }}
 ProxyPassReverse {{ localpath }} "balancer://{{balancer_name}}{{remotepath}}"
 {% endif %}
 {% else %}

roles/httpd/website/templates/website.conf

@@ -142,13 +142,18 @@
 
   ProxyRequests Off
   ProxyVia Off
+{% if site_name == 'koji.fedoraproject.org' or site_name == 'riscv-koji.fedoraproject.org' %}
+
+  # Long timeout for koji operations (completeBuild with texlive, watch-task, watch-logs)
+  ProxyTimeout 10800
+{% endif %}
 
 {% if site_name.startswith('openqa') %}
 # Openqa uses websockets, so allow them through anubis proxying.
-  ProxyPass / http://127.0.0.1:8987/ upgrade=websocket
+  ProxyPass / http://127.0.0.1:8987/ upgrade=websocket keepalive=on ttl=10
   ProxyPassReverse / http://127.0.0.1:8987/
 {% else %}
-  ProxyPass / http://127.0.0.1:8987/
+  ProxyPass / http://127.0.0.1:8987/ keepalive=on ttl=10
   ProxyPassReverse / http://127.0.0.1:8987/
 {% endif %}
 {% else %}

roles/koji_hub/templates/kojihub.conf.j2

@@ -3,6 +3,7 @@
 #
 
 KeepAlive On
+KeepAliveTimeout 15
 MaxKeepAliveRequests 0
 
 Alias /kojihub /usr/share/koji-hub/kojiapp.py