0001686: Failed to switch content type for HTTP response from UTF-8 to ISO-8859-1

1686

major

always

04-12-07 17:01

04-13-07 14:27

Reporter:

gzhu

Platform:

Dell

Assigned To:

ferg

OS:

Debian

Priority:

normal

OS Version:

2.6.7-1-686-smp

Status:

closed

Product Version:

3.0.22

Product Build:

release version

Resolution:

fixed

Projection:

none

ETA:

none

Fixed in Version:

3.1.1

Summary:

0001686: Failed to switch content type for HTTP response from UTF-8 to ISO-8859-1

Description:

We have most of our pages served up with HTTP response encoding UTF-8 and we also have some binary files served up straight from database, the servlet code was just trying to copy those raw bytes onto output stream, but that process failed with errors:

localhost ===> exception message: illegal utf8 encoding at 0xd0, cf
localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> java.io.CharConversionException: illegal utf8 encoding at 0xd0, cf
localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.vfs.i18n.UTF8Reader.read(UTF8Reader.java:97)
localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.server.connection.ToCharResponseStream.write(ToCharResponseStream.java:301)
localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.server.connection.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:68)
localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at java.io.DataOutputStream.write(DataOutputStream.java:90)

This happens to 3.0.22 and 3.0.23; and most probably 3.1.0, by reviewing the source code - resin bug 1538 prevented our applicatioin from running on Resin 3.1.0.

Steps To Reproduce:

Prepare a servlet or JSP with content type UTF-8 and another servlet to read spreadsheet data from database and served up as content type "application/vnd.ms-excel", see following code:

response.setContentType("application/vnd.ms-excel");
response.setHeader("Content-Disposition", "attachment; filename=xyz");
_out = response.getOutputStream();
// read raw binary data from database and do
_out.write(buffer, 0, len);
_out.close();
.....

To make it fail:

Access the first UTF-8 servlet and then access the spreadsheet servlet; if this does not happen the first time, repeat the process a couple of more times. You should see these exceptions soon.

Additional Information:

The reason is that the Encoding Reader in ResponseAdapter is not cleared up when the Response object was put back into FreeList. And subsequent response.setCharacterEncoding(..) will not reset Encoding Reader.

The proposed fix is to simply reset _encodingReader when it is put into FreeList.

I fixed problem by modifying 2 files: com/caucho/server/connection/ResponseAdapter.java, com/caucho/server/connection/ToCharResponseAdapter.java.

I am pretty sure the update in ResponseAdapter.java will not do harm, but I want to be assured that the change in ToCharResponseAdapter.java is OK, since I don't really understand why getResponse().getWriter().close() was commented out.

Followings are code diff:

gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection> diff -u -U 6 ToCharResponseAdapter.java.orig ToCharResponseAdapter.java
--- ToCharResponseAdapter.java.orig 2006-08-01 12:44:05.000000000 -0700
+++ ToCharResponseAdapter.java 2007-04-12 15:51:01.487023743 -0700
@@ -135,12 +135,17 @@
      */
     public void close()
       throws IOException
     {
       // jsp/1730
       flushBuffer();
+
+ // Fix!!
+ // invoking ToCharResponseStreamWrapper.close() to clear out _encodingReader
+ // this is different from getResponse().getWriter().close()
+ super.close();

       // server/172q
       // getResponse().getWriter().close();
     }

     protected void writeNext(char []buffer, int offset, int length)
gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection> diff -u -U 6 ResponseAdapter.java.orig ResponseAdapter.java
--- ResponseAdapter.java.orig 2006-08-01 12:44:05.000000000 -0700
+++ ResponseAdapter.java 2007-04-12 16:12:59.151666014 -0700
@@ -37,12 +37,13 @@
import javax.servlet.http.Cookie;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.UnsupportedEncodingException;
import java.util.logging.Logger;
+import java.util.logging.Level;

public class ResponseAdapter extends ResponseWrapper
   implements CauchoResponse {
   private static final Logger log = Log.open(ResponseAdapter.class);

   private static final FreeList<ResponseAdapter> _freeList =
@@ -339,12 +340,21 @@

   /**
    * Clears the adapter.
    */
   protected void free()
   {
+ // The Fix
+ try
+ {
+ close();
+ }
+ catch(Throwable e)
+ {
+ log.log(Level.INFO, "Failed to close ResponseStream before putting it back into FreeList", e);
+ }
     _request = null;
     _responseStream = null;

     setResponse(null);
   }
}
gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection>

System Description:

Development environment

Relationships

Attached Files:

Mantis - Resin
Viewing Issue Advanced Details

ID:	Category:	Severity:	Reproducibility:	Date Submitted:	Last Update:
1686		major	always	04-12-07 17:01	04-13-07 14:27

Reporter:	gzhu	Platform:	Dell
Assigned To:	ferg	OS:	Debian
Priority:	normal	OS Version:	2.6.7-1-686-smp
Status:	closed	Product Version:	3.0.22
Product Build:	release version	Resolution:	fixed
Projection:	none
ETA:	none	Fixed in Version:	3.1.1

Summary:	0001686: Failed to switch content type for HTTP response from UTF-8 to ISO-8859-1
Description:	We have most of our pages served up with HTTP response encoding UTF-8 and we also have some binary files served up straight from database, the servlet code was just trying to copy those raw bytes onto output stream, but that process failed with errors: localhost ===> exception message: illegal utf8 encoding at 0xd0, cf localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> java.io.CharConversionException: illegal utf8 encoding at 0xd0, cf localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.vfs.i18n.UTF8Reader.read(UTF8Reader.java:97) localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.server.connection.ToCharResponseStream.write(ToCharResponseStream.java:301) localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at com.caucho.server.connection.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:68) localhost 2007/04/12 07:58:24.967 DEBUG [6]>>> at java.io.DataOutputStream.write(DataOutputStream.java:90) This happens to 3.0.22 and 3.0.23; and most probably 3.1.0, by reviewing the source code - resin bug 1538 prevented our applicatioin from running on Resin 3.1.0.
Steps To Reproduce:	Prepare a servlet or JSP with content type UTF-8 and another servlet to read spreadsheet data from database and served up as content type "application/vnd.ms-excel", see following code: response.setContentType("application/vnd.ms-excel"); response.setHeader("Content-Disposition", "attachment; filename=xyz"); _out = response.getOutputStream(); // read raw binary data from database and do _out.write(buffer, 0, len); _out.close(); ..... To make it fail: Access the first UTF-8 servlet and then access the spreadsheet servlet; if this does not happen the first time, repeat the process a couple of more times. You should see these exceptions soon.
Additional Information:	The reason is that the Encoding Reader in ResponseAdapter is not cleared up when the Response object was put back into FreeList. And subsequent response.setCharacterEncoding(..) will not reset Encoding Reader. The proposed fix is to simply reset _encodingReader when it is put into FreeList. I fixed problem by modifying 2 files: com/caucho/server/connection/ResponseAdapter.java, com/caucho/server/connection/ToCharResponseAdapter.java. I am pretty sure the update in ResponseAdapter.java will not do harm, but I want to be assured that the change in ToCharResponseAdapter.java is OK, since I don't really understand why getResponse().getWriter().close() was commented out. Followings are code diff: gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection> diff -u -U 6 ToCharResponseAdapter.java.orig ToCharResponseAdapter.java --- ToCharResponseAdapter.java.orig 2006-08-01 12:44:05.000000000 -0700 +++ ToCharResponseAdapter.java 2007-04-12 15:51:01.487023743 -0700 @@ -135,12 +135,17 @@ / public void close() throws IOException { // jsp/1730 flushBuffer(); + + // Fix!! + // invoking ToCharResponseStreamWrapper.close() to clear out _encodingReader + // this is different from getResponse().getWriter().close() + super.close(); // server/172q // getResponse().getWriter().close(); } protected void writeNext(char []buffer, int offset, int length) gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection> diff -u -U 6 ResponseAdapter.java.orig ResponseAdapter.java --- ResponseAdapter.java.orig 2006-08-01 12:44:05.000000000 -0700 +++ ResponseAdapter.java 2007-04-12 16:12:59.151666014 -0700 @@ -37,12 +37,13 @@ import javax.servlet.http.Cookie; import javax.servlet.http.HttpServletResponse; import java.io.IOException; import java.io.PrintWriter; import java.io.UnsupportedEncodingException; import java.util.logging.Logger; +import java.util.logging.Level; public class ResponseAdapter extends ResponseWrapper implements CauchoResponse { private static final Logger log = Log.open(ResponseAdapter.class); private static final FreeList<ResponseAdapter> _freeList = @@ -339,12 +340,21 @@ /* * Clears the adapter. */ protected void free() { + // The Fix + try + { + close(); + } + catch(Throwable e) + { + log.log(Level.INFO, "Failed to close ResponseStream before putting it back into FreeList", e); + } _request = null; _responseStream = null; setResponse(null); } } gzhu-gx270:/usr/resin-3.0.22-src/modules/resin/src/com/caucho/server/connection>
System Description:	Development environment
Relationships
Attached Files: