You are viewing an historical archive of past issues. Please report new issues to the appropriate project issue tracker on GitHub.

Home » Issues » Bug #1747

Bug #1747: Wrong mime-type being used for https://wincent.dev/

Kind	bug
Product	wincent.dev
When	2010-11-28T10:34:15Z
Status	closed
Reporter	Greg Hurrell
Tags	no tags

Description

$ curl -i https://wincent.dev
HTTP/1.1 200 OK
Server: nginx/0.7.67
Date: Sun, 28 Nov 2010 10:32:13 GMT
Content-Type: application/octet-stream
Content-Length: 9770
Last-Modified: Sun, 21 Nov 2010 18:19:43 GMT
Connection: keep-alive
Accept-Ranges: bytes

This breaks the front page on Firefox and probably a lot of other browsers too. Works in Safari, which is why I hadn't noticed. I expect the culprit is something in the recent Rails 3.0.3 update, seeing as nothing else has changed on the server in the meantime.

Comments

Greg Hurrell 2010-11-28T10:36:02Z

Can't reproduce locally, either connecting via nginx or directly to the mongrel instance.
Greg Hurrell 2010-11-28T10:44:15Z
Ditto for Unicorn, locally.
Greg Hurrell 2010-11-28T10:54:11Z
As a temporary workaround I've changed the default_type in my nginx config from:
```
default_type application/octet-stream;
```
To:
```
default_type text/html;
```
And it is indeed now serving the front page with that type, thus fixing the breakage in Firefox.

Now, this shouldn't be necessary, as Rails should be setting the content type for all requests. Indeed, it is setting it for other extensionless requests (eg. https://wincent.dev/blog).

What is not clear to me is why I can't reproduce the fault locally. In my local setup nginx also has the default_type application/octet-stream setting, which is the default in all sample nginx configs I've ever seen, and it still serves the root of the site using the correct text/html type.
Greg Hurrell 2010-11-28T11:07:21Z
Another observation:
- https://wincent.dev/ is supposed to show products#index but was failing due to the MIME type thing
- https://wincent.dev/products worked fine
- there is an index.html file on the disk created via page caching (ie. in public/index.html) so Rails wasn't even been hit for those dud requests
- there is also a products.html file on the disk created via page caching (ie. at public/products.html); to be honest I am not even sure what is responsible for creating the previously mentioned index.html file
- there is not a page-cached file at public/products/index.html (although there are cached pages for the products#show action; eg. public/products/synergy.html and so on)
- a request for the site root via Safari shows up in the logs like this: [ip] - - [28/Nov/2010:06:04:06 -0500] "GET / HTTP/1.1" 200 3415 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; es-es) AppleWebKit/533.18.1 (KHTML, like Gecko) Version/5.0.2 Safari/533.18.5" "-"
Greg Hurrell 2010-11-28T11:16:35Z
Local testing in production mode (connecting directly to Unicorn):
- public/index.html gets created on first request to /
- public/products.html gets created on first request to /products
Connecting to nginx:
- mongrel gets hit every time, as in local configuration nginx isn't checking the cache files (even though I would expect it to); seems like there is a bug in the local configuration

Greg Hurrell 2010-11-28T13:46:40Z

One thing I've noticed in testing:

$ curl -I https://wincent.dev

Returns the right headers but:

$ curl -i https://wincent.dev

does not.

More detail (-v switch to show sent headers as well):

$ curl -i -v https://wincent.dev
* About to connect() to wincent.dev port 443 (#0)
*   Trying 184.73.232.208... connected
* Connected to wincent.dev (184.73.232.208) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* 	 subject: serialNumber=nVTtmHtVQS7Jq8FQvIxPDYgOFIdOCRZb; C=AU; O=wincent.dev; OU=GT09983718; OU=See www.rapidssl.com/resources/cps (c)10; OU=Domain Control Validated - RapidSSL(R); CN=wincent.dev
* 	 start date: 2010-01-08 22:12:10 GMT
* 	 expire date: 2015-02-09 00:39:03 GMT
* 	 common name: wincent.dev (matched)
* 	 issuer: C=US; O=Equifax; OU=Equifax Secure Certificate Authority
* 	 SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: wincent.dev
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx/0.7.67
Server: nginx/0.7.67
< Date: Sun, 28 Nov 2010 13:45:01 GMT
Date: Sun, 28 Nov 2010 13:45:01 GMT
< Content-Type: application/octet-stream
Content-Type: application/octet-stream
< Content-Length: 9770
Content-Length: 9770
< Last-Modified: Sun, 21 Nov 2010 18:19:43 GMT
Last-Modified: Sun, 21 Nov 2010 18:19:43 GMT
< Connection: keep-alive
Connection: keep-alive
< Accept-Ranges: bytes
Accept-Ranges: bytes

And:

$ curl -I -v https://wincent.dev
* About to connect() to wincent.dev port 443 (#0)
*   Trying 184.73.232.208... connected
* Connected to wincent.dev (184.73.232.208) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:
* 	 subject: serialNumber=nVTtmHtVQS7Jq8FQvIxPDYgOFIdOCRZb; C=AU; O=wincent.dev; OU=GT09983718; OU=See www.rapidssl.com/resources/cps (c)10; OU=Domain Control Validated - RapidSSL(R); CN=wincent.dev
* 	 start date: 2010-01-08 22:12:10 GMT
* 	 expire date: 2015-02-09 00:39:03 GMT
* 	 common name: wincent.dev (matched)
* 	 issuer: C=US; O=Equifax; OU=Equifax Secure Certificate Authority
* 	 SSL certificate verify ok.
> HEAD / HTTP/1.1
> User-Agent: curl/7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
> Host: wincent.dev
> Accept: */*
> 
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< Server: nginx/0.7.67
Server: nginx/0.7.67
< Date: Sun, 28 Nov 2010 13:46:03 GMT
Date: Sun, 28 Nov 2010 13:46:03 GMT
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< Connection: keep-alive
Connection: keep-alive
< Status: 200 OK
Status: 200 OK
< ETag: "9d7b98c51ced2c758d6057750674df1b"
ETag: "9d7b98c51ced2c758d6057750674df1b"
< X-UA-Compatible: IE=Edge,chrome=1
X-UA-Compatible: IE=Edge,chrome=1
< X-Runtime: 0.064133
X-Runtime: 0.064133
< Set-Cookie: user_id=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: user_id=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
< Set-Cookie: session_key=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
Set-Cookie: session_key=; path=/; expires=Thu, 01-Jan-1970 00:00:00 GMT
< Cache-Control: max-age=0, private, must-revalidate
Cache-Control: max-age=0, private, must-revalidate

< 
* Connection #0 to host wincent.dev left intact
* Closing connection #0
* SSLv3, TLS alert, Client hello (1):

Greg Hurrell 2010-11-28T14:27:48Z
Ok, I've finally gotten to the bottom of the issue and I believe it is now fixed definitively.

I believe the problem most likely dates back to when I added support for a "dual" web root:
- public corresponding to the standard public directory in a Rails application
- static for uploading certain generated, static documentation that should persist across deployments of different versions
This allows me to have URLs like:
- https://wincent.dev/products/wikitext: on the first request, this hits the Rails application and dynamically generates the page, which is later cached to disk
- https://wincent.dev/products/wikitext.html: subsequent requests just use this cached version
- https://wincent.dev/products/wikitext/doc: static uploaded documentation (generated by YARD), and stored on the disk under static, not public, so that it will be preserved across deployments
I believe that somewhere along the way in the implementation of this feature, serving up index.html when https://wincent.dev (or https://wincent.dev/) is requested was broken. I still don't know exactly why, but I do not what fixed it: explicitly adding in handling of index.html files:
```
set $index_filename 'index.html';
if (-f $request_filename$index_filename) {
  # URI has trailing slash
  rewrite (.*) $1$index_filename break;
}

if (-f $request_filename/index.html) {
  # URI has no trailing slash
  rewrite (.*) $1/ permanent;
}
```
Note that using nginx's built-in index directive (ie. index index.html) did not work, probably due to the complication relation among the different location blocks and rewrite rules. The whole section now looks like this (with tweaks to make sure that only GET posts are affected):
```
# rewrite all requests to back to wincent.dev
if ($host != wincent.dev) {
  rewrite ^(.*) https://wincent.dev$1 permanent;
}

# serve static content without hitting Rails
# this also bypasses our maintenance page check
location ~ /(images|javascripts|stylesheets)/ {
  expires 72h;
}

location /system/maintenance.html {
  # always allow access to this file; explained here:
  # http://www.ruby-forum.com/topic/141251
}

location / {
  if (-f $document_root/system/maintenance.html) {
    error_page 503 /system/maintenance.html;
    return 503;
  }

  # cached pages: only for GET requests
  set $cache_extension '';
  if ($request_method = GET) {
    set $cache_extension '.html';
  }

  if (-f $request_filename) {
    break;
  }

  set $index_filename 'index';
  if (-f $request_filename$index_filename$cache_extension) {
    # URI has trailing slash
    rewrite (.*) $1$index_filename$cache_extension break;
  }

  if (-f $request_filename/$index_filename$cache_extension) {
    # URI has no trailing slash
    rewrite (.*) $1/ permanent;
  }

  if (-f $request_filename$cache_extension) {
    rewrite (.*) $1.html break;
  }

  if (-f $static_root$uri) {
    root $static_root;
    break;
  }

  if (-f $static_root${uri}index.html) {
    # URI has trailing slash
    root $static_root;
    rewrite (.*) $1$index_filename break;
  }

  if (-f $static_root$uri/index.html) {
    # URI has no trailing slash
    rewrite (.*) $1/ permanent;
  }

  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header Host $http_host;
  proxy_set_header X_FORWARDED_PROTO https;
  proxy_max_temp_file_size 0;
  if (!-f $request_filename) {
    proxy_pass http://unicorn;
    break;
  }
}
```
They say that Apache rewrite rules can be tricky, but for me the way nginx rewriting works is actually a lot more opaque; see the IfIsEvil page on the nginx wiki]:

Directive "if" is part of rewrite module which evaluates instructions imperatively. On the other hand, nginx configuration in general is declarative. At some point due to users demand an attempt was made to enable some non-rewrite directives inside "if", and this lead to situation we have now
Greg Hurrell 2010-11-28T14:29:06Z
Status changed:
- From: new
- To: closed

Add a comment

Comments are now closed for this issue.

Bug #1747: Wrong mime-type being used for https://wincent.dev/

Description

Comments

Add a comment

Menu