Ensuring the response stream is read in UTF-8 charset in the HTTPClientHandler (!247) · Merge requests · OSDU Software / OSDU Data Platform / System / Lib / core / OS Core Common

Sabarish K R E requested to merge sabz/charsetFix into master Dec 25, 2023

The HTTPClientHandler code, is NOT system independent. The HTTPClientHandler reads the Response Stream of a HTTP call in the system default charset, which may NOT be UTF-8 in all systems.

This is causing problem in hosts where default charset is not UTF-8 (eg. US-ASCII). For example, This HTTPClientHandler is used by indexer to get records from storage service, and if the records contain characters like "Æ", it gets messed up when reading the response stream in a non-UTF-8 Charset.

So, this fix, ensures that the Response Stream is read in UTF-8, irrespective of what is the default charset in the system, to make it system independent. This is similar to how HTTP request is also encoded in UTF-8 when sending a request from the HTTPClientHandler already today.

Edited Dec 26, 2023 by Sabarish K R E

Ensuring the response stream is read in UTF-8 charset in the HTTPClientHandler

Merge request reports