Skip to content

Support mmap'ed edge lists #11

Open
@jpfairbanks

Description

@jpfairbanks

Sometimes you want to build a graph from an edgelist file, but don't have 2X memory lying around. So you should mmap the edgelist and then build the graph. Here is a sample implementation.

using Base.Mmap
using Base.Test
using LightGraphs

function elmmap{T}(io::IO, t::T, ne::Integer)
    el = Base.Mmap.mmap(io, Array{Int,2}, (n,2))
    return el
end


function elgraph(el::Array{Int,2}, maxv)
    g = Graph(maxv)
    for i in 1:size(el,1)
        src, dst = el[i,1], el[i,2]
        if src > nv(g) || dst > nv(g)
            add_vertex!(g, maximum(src,dst) - nv(g))
        end
        add_edge!(g, src, dst)
    end
    return g
end


n = 100
write("tmp.bin", [1:n 2*(1:n)])
fp = elmmap(open("tmp.bin", "r"), Int, n)
@test fp == [1:n 2*(1:n)]
g = elgraph(fp, 2n)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions