Charset: Introduce wp_utf8_chunks() to iterate through strings. - #9826
Charset: Introduce wp_utf8_chunks() to iterate through strings.#9826dmsnell wants to merge 1 commit into
wp_utf8_chunks() to iterate through strings.#9826Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the Core Committers: Use this line as a base for the props when committing in SVN: To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
Test using WordPress PlaygroundThe changes in this pull request can previewed and tested using a WordPress Playground instance. WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser. Some things to be aware of
For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation. |
f7e2115 to
0991d08
Compare
0991d08 to
85fb018
Compare
b52768a to
9e57e41
Compare
048705c to
a1c7281
Compare
c24e21d to
424b53a
Compare
This new generator function iterates through valid and invalid spans of bytes in a UTF-8 string. It is a convenience wrapper around the new `_wp_scan_utf8()` which aids a number of operations which should work on strings containing invalid spans of UTF-8, such as: - Identifying where the invalid spans are. - Operating on the valid portions while preserving the invalid spans. - Displaying broken strings. These operations can be useful during validation, sanitization, debugging, and processing uncontrolled inputs which may contain malformed sequences.
424b53a to
77d8e4c
Compare
|
Without a clear and compelling need to abstract this in Core I’m closing for now. It’s easy enough to create in an application-specific way. If such a need emerges we can re-open this for later consideration. |
1 participant
Trac ticket: Core-63863
See:
#9825,#9830,#9498, (#9826),#9827, #9798,#9828,#9829Introduces a new method,
wp_utf8_chunks(), for iterating through a string which might contain invalid spans of UTF-8 bytes.This function is largely a low-level function for working with text data. With it, it’s possible to operate on the valid UTF-8 portions of a string while preserving the invalid parts. For instance, this might be used to operate on improperly-constructed JSON data.