Skip to content

unescape_html - Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8 #103

Closed
#127
Closed
unescape_html - Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8#103
#127

Description

@bjarosze
bjarosze
opened on Dec 16, 2025

If unescaping string that contains non printable unicode character, unescape_html raises exception:

[51] pry(main)> CGI::VERSION
=> "0.5.0"
[54] pry(main)> CGI.unescape_html("\u0094 α")
Encoding::CompatibilityError: incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError)
from /bundle/ruby/3.3.0/gems/cgi-0.5.0/lib/cgi/escape.rb:130:in `gsub!'

Version 4.2 works as expected:

[16] pry(main)> CGI::VERSION
=> "0.4.2"
[17] pry(main)> CGI.unescape_html("\u0094 α")
=> "\u0094 α"

Activity

  1. baptistejub commented on Feb 26, 2026

    @baptistejub
    baptistejub
    on Feb 26, 2026

    I encountered the same problem and it seems it could be an aliasing issue (or an issue with the loading order of the C extension) because unescapeHTML is working (by using the C implementation I guess):

    irb(main):002> CGI.unescape_html("\u0094 α")
/ruby/4.0.0/lib/ruby/4.0.0/cgi/escape.rb:142:in 'String#gsub!': incompatible character encodings: BINARY (ASCII-8BIT) and UTF-8 (Encoding::CompatibilityError)
irb(main):003> CGI.unescapeHTML("\u0094 α")
=> "\u0094 α"

    Could be related to the changes made in #67 (previously the C extension was required before defining or aliasing unescapeHTML, now it's required after).

  2. pushcx commented on Mar 12, 2026

    @pushcx
    pushcx
    on Mar 12, 2026

    I just saw this bug (workaround). User-generated content was submitted with unprintable unicode characters to our Rails app; we were unable to display any page that excerpted that content.

  3. baptistejub commented on Mar 17, 2026

    @baptistejub
    baptistejub
    on Mar 17, 2026

    Should be fixed by #110

  4. hsbt
    added a commit that references this issue on Jun 23, 2026

    Fix unescapeHTML CompatibilityError in pure-Ruby fallback

    09970b0
  5. hsbt
    closed this as completedin #127on Jun 23, 2026

  6. hsbt commented on Jun 23, 2026

    @hsbt
    hsbt
    on Jun 23, 2026
    Member

    #110 fixed only C implementation. I added that to pure Ruby fallback at #127

  7. hsbt
    added a commit that references this issue on Jun 23, 2026

    [ruby/cgi] Fix unescapeHTML CompatibilityError in pure-Ruby fallback

    23f3148
  8. k0kubun
    added a commit that references this issue on Jun 24, 2026

    [ruby/cgi] Fix unescapeHTML CompatibilityError in pure-Ruby fallback

    c552f2b
  9. bmwiedemann
    added a commit that references this issue on Sep 29, 2026

    Update rubygem-cgi to version 0.5.2 / rev 2 via SR 1380830

    2a1ee9c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions