captsolo weblog

Archives for: February 2011, 21

2011-02-21

01:31:47, Categories: Semantic Web, Software Development

NQuad parsing using Jython

When in need to parse NQuad RDF files (e.g., the Billion Triples Challenge data files) Java folks can use the NxParser by Aidan Hogan and Andreas Harth: NxParser - Parser for NTriples, NQuads, and more.

You can also use it from Python (provided you use the Jython implementation).

Code:

import sys
sys.path.append("./nxparser.jar")
 
from org.semanticweb.yars.nx.parser import *
from java.io import FileInputStream
from java.util.zip import GZIPInputStream
 
def all_triples(fname, use_gzip=False):
    in_file = FileInputStream(fname)
    if use_gzip:
        in_file = GZIPInputStream(in_file)
 
    nxp = NxParser(in_file, False)
 
    while nxp.hasNext():
        triple = nxp.next()
        n3 = ([i.toN3() for i in triple])
        yield n3

The code above defines a generator function which will yield a stream of NQuad records. We can now add some demo code in order to see it in action:

February 2011
Mon	Tue	Wed	Thu	Fri	Sat	Sun
<< <	Current	> >>
	1	2	3	4	5	6
7	8	9	10	11	12	13
14	15	16	17	18	19	20
21	22	23	24	25	26	27
28

Search

Flickr photos
Del.icio.us bookmarks

Gallery

www.flickr.com

Go to captsolo's photostream

Misc

Syndicate this blog

RSS 1.0 (RDF): Posts, Comments
Atom 1.0: Posts, Comments

	`import sys`
	`sys.path.append("./nxparser.jar")`

	`from org.semanticweb.yars.nx.parser import *`
	`from java.io import FileInputStream`
	`from java.util.zip import GZIPInputStream`

	`def all_triples(fname, use_gzip=False):`
	`in_file = FileInputStream(fname)`
	`if use_gzip:`
	`in_file = GZIPInputStream(in_file)`

	`nxp = NxParser(in_file, False)`

	`while nxp.hasNext():`
	`triple = nxp.next()`
	`n3 = ([i.toN3() for i in triple])`
	`yield n3`