Comments on: How to Calculate Checksums for Large Data or Streams
Ok, so you probably know how to calculate checksums in REBOL. It is easy. For example, a simple checksum is:
sum: checksum data
For a stronger checksum, you can use a secure hashing algorithm, such as SHA1 with code like this:
sum: checksum/secure data
But, what happens if your data is too large to fit in memory, or you are streaming data that needs to be checked on the fly?
The answer is: checksum ports.
A checksum port is like any other REBOL port. Here's a quick example of how to compute an ongoing (streaming) checksum:
sport: open [scheme: 'checksum]
insert sport "this is some text"
insert sport "this is more text"
insert sport "this is the final text"
insert sport ...
update sport
sum: copy sport
close sport
print sum
The update function is required to compute the final checksum value. Without it, the copy function will return none.
The default algorithm for the checksum above is SHA1. The open line above is equivalent to writing:
sport: open [scheme: 'checksum algorithm: 'sha1]
If you want MD5 instead, use this line:
sport: open [scheme: 'checksum algorithm: 'md5]
I hope you find this checksum technique of value for your networking and file operations.
14 Comments Comments:
Brian Hawley
13-Jun-2006 12:39 |
The question of how to do this has been around for a while. Are these checksum ports new or have they just been undocumented thus far? | Maxim Olivier-Adlhoch
13-Jun-2006 13:08 |
Here is my solution to the documentation issue:
clone-carl: :launch
until[
clone-carl compose ["document/only " (next feature in REBOL)]
documented?/all REBOL
]
:)
the only issue is that this might be an infinite loop! | Jaime
13-Jun-2006 14:44 |
Having checksum as port requires plumbimg to feed the port. I'm not sure that is the best approach.
It also seem strange that the port system will persist in R3, It was my understanding that the ports systems was going to be redone.
It also strange that the same feature will have two different forms checksum as function and checksum as port. This may obscure things for newbies. | GreG
13-Jun-2006 16:06 |
I like it. | Carl Sassenrath
13-Jun-2006 18:02 |
Brian: yes, it's been there for a while. Recently, someone asked how to use it, and Google did not show any obvious doc on it, so I decided to add a quick blog note on it.
Maxim: yes, a cloning machine, but if not, then just some other REBOLers joining in would be a great start.
Jaime: What makes you think that the port system will presist in R3? Your understanding what correct: it is redone. On the different forms: I suppose you can think of the function as a shortcut. | Dan Lee
13-Jun-2006 23:39 |
This is very useful to me. Thanks! | Petr Krenzelok
14-Jun-2006 1:32 |
:)
Carl,
what about streamed 'parse? :)
Petr | Anton Rolls
14-Jun-2006 1:36 |
Of value?! This is a great simplicity.
I just needed this the other day and had no idea it was possible this way. I almost went and translated a C source to rebol.
Is there a way to list the available schemes ?
This is not common knowledge.
Thanks for posting the example,
Regards, | Cal Dixon
14-Jun-2006 2:37 |
Thanks, I've been needing this feature for quite a while now. | Maxim Olivier-Adlhoch
14-Jun-2006 15:50 |
Carl: hehe, I meant specifically documenting all the neat stuff that probably lies hidden within REBOl.
(all the stuff in your head)
WRT helping... there is this initiative:
http://rebdocproj.org/?lang=en
sorry to all for being OT... my last post. promise. | Edoc
14-Jun-2006 17:06 |
"... some other REBOLers joining in would be a great start."
REBOL has foundational docs, but its educational/practical knowledge is incomplete and scattered across different sites, forums/wikis and mailing lists.
I think the availability and quality of docs is related to the supply of REBOL gurus. The "zen" factor of REBOL that makes doc writing a more challenging proposition.
Ideally a doc-writer would like to print with authority, "This is the meaning and intent of this function/feature; these are the cases it was designed to handle; here are some of the limitations and hidden gotcha's, and here's how we build something pretty cool with this..."
If this kind of info isn't made available to people, then a quality doc probably won't be written until Carl writes it himself. It's a self-reinforcing cycle: a programmer failed to develop into a REBOL guru, so he/she didn't write a doc, so other REBOL programmers failed to become REBOL gurus, and didn't write docs, etc..."
If the art of REBOL is best learned through bootstrapping, I think most programmers will seek enlightenment elsewhere. So, gurus, don't leave it all on Carl. Please write docs on interesting projects. Show specific coding details about how you created something cool. Explain it so that someone else can recreate or extend it.
Edoc
"Plant a new Truffula. Treat it with care.
Give it clean water. Feed it fresh air.
Grow a forest. Protect it from axes that hack.
Then the Lorax and all of his friends may come back." | Robert Lancaster
19-Jun-2006 22:35 |
:)
I'm with Petr, Streamed parse!
I've been wanting that for a loooong time....
why then I'd be able to connect a serial: port! to a parse: port! and things would seem to magically happen...
( Skips off to imaginary programmer utopia with huge smile on face ) | Anonymous
22-Jun-2006 0:16 |
forgive the ignorence, but would this allow for instance to open a fifo (of fixed size)port on windows instead of a growing file, and pipe a DVB-T channel/raw.TS through that and pull off the raw pids etc to muticast that selection off to other ip's, checking for basic errors when you want?. | Christophe
22-Jun-2006 0:16 |
:)
AS Maxim stated, the REBOL Documentation Project aims to serve as a central documentation point for any REBOL-related documentations.
Those can be published on the site, or we can publish a comment and a reference to another site.
Everybody is welcome to participate to this documentation effort, which is having a great success within the French-speaking community.
FYI, the site has get over the 4000 visits since February last year, with a actual mean of 92 visits a day. Just serving REBOL visibility for free :)
Meet us there: http://rebdocproj.org |
Post a Comment:
You can post a comment here. Keep it on-topic.
|
