Identify Encryption & Hashing Characteristics

What is Encryption.

Encryption uses math to protect sensitive electronic information. In cryptography, encryption is the process of converting information or data into a code, designed to prevent unauthorized access. This process converts the original representation of the information, known as plaintext, into an alternative form known as ciphertext. Encryption does not itself prevent interference but denies the intelligible content to a would-be interceptor.

History

Ancient

One of the earliest forms of encryption is symbol replacement.One of the most famous military encryption developments was the Caesar Cipher, which used a simple substitution method where letters in the alphabet are shifted by a designated fixed number of spaces to yield an encoded alphabet.

19^th–20^th century

Around 1790, Thomas Jefferson theorized a cipher to encode and decode messages in order to provide a more secure way of military correspondence.

A similar device to the Jefferson Disk, the M-94, was developed in 1917 independently by US Army Major Joseph Mauborne. This device was used in U.S. military communications until 1942.

In World War II, the Axis powers used a more advanced version of the M-94 called the Enigma Machine. The Enigma Machine was more complex because unlike the Jefferson Wheel and the M-94, each day the jumble of letters switched to a completely new combination.

Modern

Today, encryption is used in the transfer of communication over the Internet for security and commerce. As computing power continues to increase, computer encryption is constantly evolving to prevent eavesdropping attacks.

Key Objective of Encryption Data

• Confidentiality: Encryption ensures that only authorized parties can get access to data and recognize the information.
• Data Integrity: Encryption can also provide data integrity by making sure that the encrypted data remains unchanged during transmission. Any unauthorized changes to the encrypted information will render it undecipherable or will fail integrity checks.
• Authentication: Encryption may be used as part of authentication mechanisms to verify the identification of the communication party.
• Non-Repudiation: Through encryption, events can make sure that individuals who handle data cannot deny their involvement in growing or sending selected pieces of data.

Quantum-Resistant Cryptographic Algorithms

As of July 2022, The U.S. Department of Commerce’s National Institute of Standards and Technology (NIST) has chosen the first group of encryption tools that are designed to withstand the assault of a future quantum computer, which could potentially crack the security used to protect privacy in the digital systems we rely on every day. The four selected encryption algorithms will become part of NIST’s post-quantum cryptographic standard.

For general encryption used when we access secure websites, NIST has selected the CRYSTALS-Kyber algorithm. Among its advantages are comparatively small encryption keys that two parties can exchange easily, as well as its speed of operation.

For digital signatures NIST has selected the three algorithms CRYSTALS-Dilithium, FALCON and SPHINCS+. NIST recommends CRYSTALS-Dilithium as the primary algorithm, with FALCON for applications that need smaller signatures than Dilithium can provide. The third, SPHINCS+, is somewhat larger and slower than the other two, but it is valuable as a backup for one chief reason: It is based on a different math approach than all three of NIST’s other selections.

Hashing

Hashing takes any kind of data (text, number, file) and transforms it into a fixed-size alphanumeric string called a hash. This hash acts like a unique fingerprint for the original data.

This transformation is done by a special mathematical function called a hash function. Different data will (almost always) result in different hashes.

Often The generated hashes are then stored in a special data structure called a hash table. This table acts like a giant index, allowing for faster lookups.

How hashing works

Input data is key

The input data to be hashed is also known as the key. A key can be in just about any format. It could be a string of text, a list of numbers, an image, or even an application file.

The hash function

The central part of any hashing process is the hashing function, which takes the key and converts it to a fixed-length string of characters. - Each hashing algorithm produces it's own unique length of hash. - Ex: MD5 hash = 128 bits, or 32 characters

The hash value

The final output of the hash function is the hash value, which ideally should be unique to each input.

Properties of hashing algorithms

• Deterministic
• If you use the same data input and the same hashing algorithm, then the final hash value should be the same.
• Quick computation
• An effective hashing algorithm quickly processes any data type into a unique hash value.
• Irreversible
• Hash functions are designed to make it impossible to regenerate a message or file from the hash value.

Hash Collisions

Although rare, when two different pieces of data using the same hashing algorithm generate the same hash, it is called a collision.

Message Digest Algorithm 5(MD5)

MD5 is a widely used cryptographic hash function that produces a 128-bit (16-byte) hash value, typically represented as a 32-digit hexadecimal number. It is commonly used for tasks such as data integrity verification and digital signatures.

MD5 was once widely used in various applications. However, vulnerabilities were discovered in MD5 over time, particularly related to collision attacks. Due to these vulnerabilities, MD5 is now considered cryptographically broken and unsuitable for cryptographic purposes where strong collision resistance is required.

$ echo -n "Message Digest Algorithm 5"| md5sum
b6bf6cf383fc864fce5c2894b4001992

MD5 Collision Example

These two images have the same md5 hash. 253dd04e87492e4fc3471de5e776bc3d

Secure Hash Algorithm 1(SHA1)

Secure Hash Algorithm 1, is a specific hashing function developed by the National Institute of Standards and Technology (NIST). It takes an input of any size and produces a unique 160-bit (20-byte) fingerprint called a message digest.

SHA-1's vulnerability to collisions weakens its ability to ensure data integrity. An attacker could potentially: - Forge digital signatures: By creating a collision with a signed document, they could create a malicious file with a valid-looking signature. - Tamper with data undetected: If data is verified using SHA-1, a collision could allow attackers to modify the data without affecting the hash, making the change appear legitimate.

Due to these security concerns, SHA-1 is no longer considered secure for most applications. NIST (National Institute of Standards and Technology) recommends using stronger hashing algorithms like SHA-256 or SHA-3.

$ echo -n "Safeguarding digital assets through proactive defense and vigilant monitoring." | sha1sum 
b77e6631d550cf7f3407529dc84ec99eafc30170 

SHA1 Collision Example

You can download the two messages below, and verify their hash with the sha1sum tool. Use Hexdump -C to verify the differences in Data.

• messageA
• messageB

$ hexdump -C messageA 
$ hexdump -C messageB
$ sha1sum messageA messageB

Secure Hash Algorithm 2(SHA2)

SHA-2 (Secure Hash Algorithm 2) is a set of cryptographic hash functions designed by the United States National Security Agency (NSA) and first published in 2001. SHA-2 includes significant changes from its predecessor, SHA-1. The SHA-2 family consists of six hash functions. SHA-256 and SHA-512 are the most commonly used out of the six and are novel hash functions computed with eight 32-bit and 64-bit words, respectively.

echo -n "Uniting skill and strategy to dominate the virtual battlefield and showcase military excellence" |sha256sum 
614b2dcb025b01ce1e8030d039d63812f6b0a5442207e9a987523f8976ef53f2

echo -n "Empowering tomorrow's leaders through disciplined learning, resilience, and service to uphold the values of the army." |sha512sum 
9b2df263bfef0755106d1f4476120f860472220d4a0f73ab83a1c01c2309aefa42f8d8fcc2a20a1226be769959543a37615b6dac7e083f3ea2d53ea12695d02d

Hashing is not Encryption

Hashes cannot be easily reversed to obtain the original data. They are for verification, not security. Encryption scrambles data to protect confidentiality, while hashing ensures data hasn't been modified.

Incredibly Important and Useful Hashing Commands

How to grep for a hash in a directory (without drilling into other directories):

md5sum * | grep 'hash_string'

How to grep for multiple hash values in a single directory:

md5sum * | grep -E 'hash_1|hash_2'

How to search nested directories for a particular hash value:

find . -type f | xargs md5sum | grep 'hash_string'

How to search nested directories for files with matching hash values:

find . -type f | xargs md5sum | sort | uniq -D -w32

NOTE: A lowercase -d with uniq will mess you up.