Comments on: The Problem with /Direct Port Access
There has been a long standing problem with /direct access I/O ports when used for random access (seeking) within files. This has made access to large files difficult.
Let me illustrate the problem with an example:
write %example.txt "1234567890" port: open/direct/binary %example.txt print as-string copy/part port 2 12 print as-string copy/part port 2 34 port: head port print as-string copy/part port 2 56
As you can see, the head function did not have any effect on the result. It was ignored. You can also see that the copy function is automatically advancing the read position within the file.
This behavior is different from ports opened without /direct access (and it is also different from the standard REBOL concept of a series). This example shows why:
port: open/binary %example.txt print as-string copy/part port 2 12 print as-string copy/part port 2 12 port: skip port 2 print as-string copy/part port 2 34 port: head port print as-string copy/part port 2 12
Here without the /direct mode, the port acts just like a standard REBOL series (as you would expect).
What makes the problem worse is that the /direct mode does not perform an actual seek on the file. If you open a large file (say 100 MB) and skip 50 MB forward, the direct port will actually perform a read of all 50 MB of data to reach the desired position. That slows down random access within large files. (And skip may also hang at times.)
In my opinion, the implementation of /direct mode on files is broken. The question is how should it be fixed. There are two choices:
Choice 1 seems like the perfect solution, but there may be quite a few applications that open files in direct mode, and they will break if they expect the file position to move forward after each copy function.
So, it looks like we will need to add a /seek mode to ports. We did this in the new asynchronous version of REBOL, and it worked well. But the current releases use a different implementation of ports, so we will need to design a new solution.
We've raised this issue to a high priority bug fix, because REBOL absolutely needs to have a better way to randomly access large files. You can look for a new beta test release of REBOL very soon.
Post a Comment:
You can post a comment here. Keep it on-topic.