Uploaded image for project: 'Qt'
  1. Qt
  2. QTBUG-19335

Qt's Shift-JIS codec behaves different from ICU with regards to the backslash and tilde ascii characters

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • P2: Important
    • 4.8.0
    • 4.7.3
    • None
    • 8e321cd869da7ff1cf0168da41aa0246b44867cc

    Description

      When using the Shift-JIS codec in Qt to convert to Unicode, the characters 0x5c and 0x7e are mapped to the yen and overline symbols - according to JIS X 2010.

      ICU on the other hand maps these characters to backslash and tilde - ascii compatible.

      We should adjust Qt to behave like ICU with respect to that.

      This issue becomes particularly visible with WebKit, where other WebKit browsers are using ICU. A real-world example is for example the site http://www.bidders.co.jp/ - which specifies shift-jis as encoding and loads jquery via a script tag. At that point it is assumed that jquery is encoded
      in shift-jis. JQuery uses regular expressions, which in turn commonly contain backslash characters. With icu these are preserved and the regular expression
      parsing proceeds normally. With Qt the backslash is mapped to yen (unicode yen 0x00a5), which breaks parsing.

      In addition there's a WebKit layout test that fails due to this difference: https://bugs.webkit.org/show_bug.cgi?id=39852

      The attached test-case demonstrates the difference. It can be compiled with g++ -o test main.cpp `pkg-config --cflags --libs QtGui` `icu-config --cflags --ldflags`

      Attachments

        1. main.cpp
          0.7 kB
        2. patch.txt
          2 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            shausman Simon Hausmann
            shausman Simon Hausmann
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes