On 11/3/2009 9:29 AM, Johannes Graumann wrote:> Hi,
>
> I'm dealing which huge files I would like to index. On a linux system
"grep
> -buo <PATTERN> <FILENAME>" hands me the byte offsets for
"PATTERN" very
> quickly and I am looking to emulate that speed and ease with native R tools
> - for portability and elegance. "gregexpr" should be able to do
that but I
> fail to combine it with "scan" or an equivalent to parse the
whole file
> without having to read it all into memory.
I think you are going to have to write this yourself. R doesn't have
very many stream oriented functions: almost everything is aimed at
having the whole thing in memory.
You will also have trouble with the byte offsets. The semantics of the
-u option to grep are quite strange (at least according to the man page
on Cygwin).
What I'd do given your problem is use readLines to read the file, then
post-process the result of gregexpr to give line and byte offset pairs
for each match; those are more useful in R than the rather bizarre "byte
offsets" that grep -buo will give. But for a huge file you'll probably
have to do this in blocks, as the whole file may be too big.
Duncan Murdoch
>
> I'd be grateful for any hints on how to do this without a
"pipe("grep -buo
> <PATTERN> <FILENAME>")".
>
> Thanks, Joh
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.