hw-json-simd

SIMD-based JSON semi-indexer

https://github.com/haskell-works/hw-json-simd#readme

LTS Haskell 23.28:0.1.1.3
Stackage Nightly 2025-05-05:0.1.1.2@rev:1
Latest on Hackage:0.1.1.3

See all snapshots hw-json-simd appears in

BSD-3-Clause licensed by John Ky
Maintained by [email protected]
This version can be pinned in stack with:hw-json-simd-0.1.1.3@sha256:6e79ac7644137aa1079dfc6775ee584980cf4878db35a98c5952b1f4a6882f6f,5677

Module documentation for 0.1.1.3

Used by 1 package in lts-23.28(full list with versions):
hw-json-simd

hw-json-simd

This library/tool will generate semi-indexes on JSON files as per the paper: Semi-Indexing Semi-Structured Data in Tiny Space.

The command line tool

For a given JSON file, hw-json-simd will generate two semi-index files, which both together can be loaded into a single in-memory semi-index.

The semi-index files can be generated using two methods, which will be called standard and simple, which correspond to sections 4 and 5 of the Semi-Indexing Semi-Structured Data in Tiny Space paper respectively.

Navigation of the JSON text using the standard index is suppored by the hw-json project for more information. There is currently no support for navigation of the JSON text using the simple index.

Using on the command line

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method standard

cat test.json | pv -t -e -b -a | time hw-json-simd create-index \
  -i /dev/stdin
  --output-ib-file test.json.ib.idx
  --output-bp-file test.json.bp.idx
  --method simple

Installation

From hackage

cabal new-install hw-json-simd

Changes

Changelog for hw-json-simd

Unreleased changes