Development v1.01 to Master

README.md

@@ -1,23 +1,23 @@
 # Hatnotation
 
-> Hatnotation is short for the *Hatzakis Base 64 notation system* which is a method to encode/decode arbitrary binary strings of data, invented by Steven Hatzakis and open-sourced here under Apache License 2.0 https://github.com/hatgit/hatnotation/blob/master/LICENSE.
+> Hatnotation is short for the *Hatzakis Base 64 notation system* which is a method to encode/decode arbitrary binary strings of data, invented by Steven Hatzakis and open-sourced here under [Apache License 2.0](https://github.com/hatgit/hatnotation/blob/master/LICENSE).
 
  <blockquote>
-<pre><code>[Use of Hatnotation is subject to Apache License 2.0 ] <a href="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/hatgit/hatnotation/blob/master/LICENSE">https://github.com/hatgit/hatnotation/blob/master/LICENSE</a>
+<pre><code>[Use of Hatnotation is subject to Apache License 2.0] <a href="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/hatgit/hatnotation/blob/master/LICENSE">https://github.com/hatgit/hatnotation/blob/master/LICENSE</a>
 </code></pre>
 </blockquote>
 
 ## Purpose
-An encoding/decoding method that allows users to compress their human-readable data into fewer humand-readable characters than other popular notation systems, for any arbitrary underlying machine-readable binary string. 
+An encoding/decoding method that allows users to compress their human-readable data into fewer human-readable characters than other popular notation systems, for any arbitrary underlying machine-readable binary string. 
 
 
 ## Warning: 
 
-This software is still in its experimental phase and should not be relied upon for production. 
+This software is still in its experimental phase and should not be relied upon for production.  
 
 ## Background on Mnemonics (private keys) and Human vs Machine-readable code
 
-Mnemonics (aka recovery phrases) are used in many popular crypto wallet applciations including BIP39 (which follows a specific wordlist and checksum requirement, among other steps for wallet derivation such as BIP32, BIP44) enable a user to backup their initial entropy in human-readable format. 
+Mnemonics (aka recovery phrases) are used in many popular crypto wallet applciations including [BIP39 (https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki) (which follows a specific wordlist and checksum requirement, among other steps for wallet derivation such as BIP32, BIP44) enable a user to backup their initial entropy in human-readable format. 
 
 For example, instead of a user having to backup a string of 128 bits or their private key, they can simply store the encoded mnemonic which represents those bits or a private key. 
 
@@ -32,7 +32,7 @@ For example, instead of a user having to backup a string of 128 bits or their pr
 |Total Words     |`132/11 = 12 words`            |`264/11 = 24 words`          |
 |----------------|-------------------------------|-----------------------------|
 
-In terms of actual pre-image resistance, the initial entropy should be generated in a cryptographically-secure manner that is pre-image resistant and resistant to other attacks, such as outlined in the W3C Cryptography API or via the "secrets" module in Python), and the psuedo-random binary string that results will be that is machine-readable where the purpose of encoding it into a mnemonic is to make it easier to notate, recite, read, and write, compared to binary (machine-readable).
+In terms of actual pre-image resistance, the initial entropy should be generated in a cryptographically-secure manner that is pre-image resistant and resistant to other attacks, such as outlined in the [W3C Cryptography API](https://www.w3.org/TR/WebCryptoAPI/) or via the [secrets](https://docs.python.org/3/library/secrets.html) module in Python), and the psuedo-random binary string that results will be machine-readable where the purpose of encoding it into a mnemonic is to make it easier to notate, recite, read, and write, (i.e. human-readable) compared to binary (machine-readable).
 
 
 
@@ -62,15 +62,18 @@ The Hatnotation system is *not intended to be an alternative to human-readable m
 Just as a mnemonic that represents a 132 bits of some initial entropy should convey 128 bits of security if generated properly, as the last 4 bits are deterministically derived from the hash-based checksum computation (hashing the initial entropy as a byte array). Those same 132 bits can be encoded using the Hatnotation system which will result in 22 characters, sourced from the 64 character library. Mathematically, 64^22 == 2^132, hence why there is no information/security loss, for example. 
 
 ### Lemma 
-For any arbitrary binary (base-2) string x of length n, after hatnotation is applied to x, the length n = ((x - (x % 6))/6)+(x %6, optionally in some cases depending on how the encoder/decoder is constructed). Optimal compression will occur when the [COMPLETE THIS SECTION]  
+For any arbitrary binary (base-2) string x of length n, after hatnotation is applied to x, the length n = ((x - (x % 6))/6)+(x %6, optionally in some cases depending on how the encoder/decoder is constructed). 
 
-- `(128-(128 % 6)) / 6 == 21 == 2^128`
-- `(132-(132 % 6)) / 6 == 22 == 2^132`
-- `(256-(256 % 6)) / 6 == 42 == 2^256`
-- `(264-(264 % 6)) / 6 == 44 == 2^264`
+Optimal compression will occur when the x modulo 6 is equal to zero, and least optimum when x modulo 6 is equal to 5 (The assumption in this last sentence should be checked). 
 
+Below are two examples of when notation is optimum, using 132 bits and 264 bits as an example, and which are standard bit-lengths for the underlying entropy that represents 12-word or 24-word mnemonics used with crypto wallets. 
 
-|  Private Key          |12-word mnemonic               |24-word mnemonic     |
+- `(64^((132-(132 % 6)) / 6) == 2^132`
+
+- `(64^((264-(264 % 6)) / 6) == 2^264`
+
+
+|  Private Key          |Length after Hatnotation     | BIP39 checksum    |
 |----------------|-------------------------------|-----------------------------|
 | 128 bits       |21 characters                |Checksum must be computed  |
 | 132 bits       |22 characters             |Checksum included     |
@@ -139,7 +142,7 @@ Using the binascii libary in python which contains the string library, we source
 47 | "101111", | ",",
 48 | "110000", | "-",
 49 | "110001", | ".",
-50 | "110010", | "/",
+50 | "110010", | "{",
 51 | "110011", | ":",
 52 | "110100", | ";",
 53 | "110101", | "<",
@@ -148,22 +151,25 @@ Using the binascii libary in python which contains the string library, we source
 56 | "111000", | "?",
 57 | "111001", | "@",
 58 | "111010", | "[",
-59 | "111011", | "\",
+59 | "111011", | "}",
 60 | "111100", | "]",
 61 | "111101", | "^",
 62 | "111110", | "_",
 63 | "111111", | "`",
 
 ## Library verification
 
-In Python version 3.7 using the strings library, the following steps can be taken to verify the library and character order and by omitting the last 4 characters "{|}~" in the fourth step below (zeroed-index values 64-67 which have been omitted from the above list as well): 
+In Python version 3.7 using the strings library, the following steps can be taken to verify the library and character order and notice that the following 4 characters are omitted "\\~/|" in the fourth step below: 
 
 - ```>>> import string```
 - ```>>> dir(string)['Formatter', 'Template', '_ChainMap', '_TemplateMetaclass', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_re', '_string', 'ascii_letters', 'ascii_lowercase', 'ascii_uppercase', 'capwords', 'digits', 'hexdigits', 'octdigits', 'printable', 'punctuation', 'whitespace']```
 - ```>>> print(string.digits+string.ascii_uppercase+string.punctuation)```
 - ```0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-./:;<=>?@[\]^_`{|}~```
 
-(i.e. ~~```"{|}~"```~~)
+- Note, the backslash `\` and forwardslash `/`characters were swapped with opening `{` and closing `}` curly brackets in the following issue: https://github.com/hatgit/hatnotation/issues/3. 
+- The list of valid Hatnotation library characters are thus as follows: 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&'()*+,-.{:;<=>?@[}]^_`
+
+(and the following four remain excluded/reserved ~~```"\/|~"```~~).
 
 ## Requirements
 
@@ -174,8 +180,31 @@ Python 3 or higher
 
 ## Tests: 
 
-## Reources: 
+> Example Test strings: 
+
+- Decode Target: `HELLOWORLD
+
+- Each letter decodes to respective 6-bit group: `"010001","001110","010101","010101","011000"," ", "100000","011000","011011","010101","001101",
+
+- Each Word as Continous string `"010001001110010101010101011000" "100000011000011011010101001101"
+
+- Concatenation of both words into one string: `"010001001110010101010101011000100000011000011011010101001101"
+
+- Converted binary string to hex (can be used as starting point to encode to "helloworld": `0x44e55562061b54d`
+
+
+Second Test: Decoder should be able to take a single backslash `\` and return index value 59, as well as multiple ones '`\\\\\`' and not be affected by any formatting/rendering issues of the app or interface (i.e. python or javascript/html).
+
+
+## Resources: 
+
+- Other base64 encoding schemes: https://en.wikipedia.org/wiki/Base64
+- Notational conventions: https://tools.ietf.org/html/rfc2822
+- Human-readable keys (1994) https://tools.ietf.org/html/rfc1751
 
 ## Roadmap/Plans:
 
+- develop a range of potential use cases 
+- potentially propose a request for comments (RFC) for consideration as a standard.
+- add error message for invalid characters (i.e. lowercase and reserved characters `\|/~`