Bug #1190: Very long form content gets truncated during post

Comments

1. Greg Hurrell 2009-01-12T06:02:32Z

   According to the docs (http://wiki.codemongers.com/NginxHttpCoreModule#client_max_body_size), the max client body size (client_max_body_size) defaults to 1 meg, which should be more than large enough to handle the post in question.

   The post in question had about 81K characters, and got cut off at around the 44K mark.

2. Greg Hurrell 2009-01-12T06:05:42Z

   Excess from access_log at time of last request:

   85.53.13.249 - - [12/Jan/2009:05:44:30 -0500] "POST /issues/1189/comments HTTP/1.1" 302 102 "http://example.com/issues/1189" "
   Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; es-es) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1" "-"
   85.53.13.249 - - [12/Jan/2009:05:44:30 -0500] "GET /issues/1189 HTTP/1.1" 200 27425 "http://example.com/issues/1189" "Mozilla/
   5.0 (Macintosh; U; Intel Mac OS X 10_5_5; es-es) AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1" "-"

   No evidence of misbehaviour...

3. Greg Hurrell 2009-01-12T11:53:48Z

   There is a size-limit in MySQL itself because Rails creates TEXT type columns, which have a limit of 65535 bytes. Witness this experiment in the development environment:

   $ script/console
   >> i = Issue.first
   >> c = i.comments.create
   >> c.body = "x" * 70000
   >> c.body.length
   => 70000
   >> c.reload
   >> c.body.length
   => 65535

   But I am not sure if that's what I'm running into here; look at this in the production database:

   >> Comment.find(4051).body.length
   => 65535

   Er, ok. That's definitely it. Not sure why BBEdit was telling me that the text was cut off at the 44K mark; it's UTF-8 but all in the ASCII range, so one byte per character, so I guess BBEdit only counts "letters" and not spaces and newlines.

   Looks like I'll have to do a migration to MEDIUMTEXT type.

4. Greg Hurrell 2009-01-12T12:40:47Z

   Doing a manual test run of the migration in the development environment on the local test machine:

   change_column :comments, :body, :text, :limit => 16777215

   Actual SQL on running rake db:migrate:

   ALTER TABLE `comments` CHANGE `body` `body` text(16777215) DEFAULT '' NOT NULL

   db/schema.rb before:

   t.text     "body",                                   :null => false

   After:

   t.text     "body",                :limit => 2147483647,                    :null => false

   Console test:

   >> c.body = "x" * 100_000
   >> c.body.length
   => 100000
   >> c.save
   >> c.body.length
   => 100000
   >> c.reload
   >> c.body.length
   => 100000

5. Greg Hurrell 2009-01-12T18:20:31Z

   In the end the migration I ended up to stop MySQL from rolling over into the LONGTEXT column type was:

   change_column :comments, :body, :text, :limit => 262143

   Definitely enough to push us into MEDIUMTEXT territory, but not enough to accidentally roll on into LONGTEXT if MySQL decides to multiple the limit by 3 because the table uses UTF-8 encoding.

6. Greg Hurrell 2009-01-12T18:21:50Z

   In any case, all fixed now locally. Will test in the staging environment and deploy to production soon. Marking as CLOSED.

Add a comment

Comments are now closed for this issue.