MD5 vs. SFV

Discuss anything related to hkSFV.

Moderators: Big-O Ryan, Big-O Mark

Post Reply
quantum

MD5 vs. SFV

Post by quantum »

I see that MD5 has a much larger checksum value. Does anyone know what the probability is that a large file can have a byte modified and have SFV not detect it? How does the SFV probablity compare to MD5?
User avatar
Big-O Ryan
Developer
Developer
Posts: 612
Joined: Fri Oct 19, 2001 11:00 pm
Location: Big-O Software
Contact:

Post by Big-O Ryan »

I'm not sure if the probability for reaching a false positive file identification is really going to tell you much.. Even for crc-32 (sfv), the probability of failure is very, very small. For md5 it is obviously much, much smaller.

Neither algorithms are perfect - the simple fact of the matter is that there are only 2^32 crc-32 codes, and that means it can only accurately (uniquely) hash 2^32 different files. MD5 can accurately assign unique hashes to 2^128 files. Both of these statements are assuming these algorithms work ideally, which they do not (SHA-1 is now considered more secure than md5). Obviously your chances are substantially better with MD5 - but your success with either will be pretty good.

Depending on your use, you may eventually encounter two files with the same crc-32 code - it is very unlikely that you will do so with MD5. In fact, many websites use MD5 codes as unique identifiers for images/documents - if they were come across two files with the same md5, this system would fail, but it seems to hold up well.

One of the saving graces of algorithms like these, when used for file verification, is that some files have specific characteristics that define a 'valid' file, in addition to its md5/crc hash. So, while your chances of changing a byte and having a file result in the same md5 are very, very low, the chances of you being able to change a byte of a zip file, and have it both result in the same md5, and still be a valid zip (or rar, or exe) file are astronomically low. Unfortunately, most files do not have validity checks like this (mp3, mpg, jpg, etc - do not).

From a security standpoint, crc-32 isn't really considered, and thus I don't see much research information on it, but you won't have too much trouble digging up research on MD5..
-Ryan
Big-O Software
Quantum

Post by Quantum »

Thanks for the informative response. My reason for asking is I have recently started added checksum files to my DVD-R and CD-R disks. I don't need the checksums to be unique, just accurate as possible in reporting a defective file.

Creating MD5 checksums doesn't seem to take any additional time, so I started going that way instead of SFV. From your response it seem clear either method will be adequate for my needs, but why not use the more accurate method if it doesn't take more time.
Guest

Post by Guest »

Quantum wrote:Thanks for the informative response. My reason for asking is I have recently started added checksum files to my DVD-R and CD-R disks.
You can use FastSum from www.fastsum.com
User avatar
Nathan
Extreme Groupie
Extreme Groupie
Posts: 723
Joined: Fri Sep 12, 2003 9:41 pm
Location: CA
Contact:

Post by Nathan »

...more than likely quantum is dead now... like this thread was...
Bethany is cool
Plasma2002b wrote:ill open YOUR port.....!
User avatar
Robpol86
Veteran
Veteran
Posts: 475
Joined: Tue Dec 03, 2002 11:48 pm
Contact:

Post by Robpol86 »

Quantum - Posted: Fri Nov 22, 2002 8:37 am Post subject:
Guest - Posted: Fri May 07, 2004 5:10 pm Post subject:

this genius sux @ reading
googabok

i'm glad

Post by googabok »

well i'm glad he said that cuz i was looking for a decent .sfv and .md5 file checker and i tried that one and it's really nice... if anyone has any other suggestions for file checkers that support .sfv and .md5 files and recursive directory support during creation and verification, it'd be nice to hear about it... :)
User avatar
Nathan
Extreme Groupie
Extreme Groupie
Posts: 723
Joined: Fri Sep 12, 2003 9:41 pm
Location: CA
Contact:

Post by Nathan »

we got to give them credit for actually using the search option though...
Bethany is cool
Plasma2002b wrote:ill open YOUR port.....!
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 1 guest