skylighting
syntax highlighting library
https://github.com/jgm/skylighting
Version on this page: | 0.7.7 |
LTS Haskell 22.37: | 0.14.3 |
Stackage Nightly 2024-10-11: | 0.14.3 |
Latest on Hackage: | 0.14.3 |
skylighting-0.7.7@sha256:0339943c68b761f45cd046b32975224744ebfdcbf5dafd55e0364f412f490b7f,10194
Module documentation for 0.7.7
skylighting
Package | License | Status |
---|---|---|
skylighting |
||
skylighting-core |
A Haskell syntax highlighting library with tokenizers derived from KDE XML syntax highlighting descriptions.
A command-line highlighter program, skylighting
, is also provided.
This project is divided up into two packages:
skylighting-core
: this provides KDE XML parsing, data types, and output formatters. This includes the core functionality of the skylighting project licensed under the BSD3 license, along with the KDE XML files, some of which are licensed under the LGPL or GPL. This package does not provide any built-in parsers corresponding to the XML descriptions, however. For that, useskylighting
.skylighting
: this exposes theskylighting-core
API but also provides bundled Haskell parser modules derived from the XML descriptions in thecore
package. This package is entirely licensed under the GPL.
Motivation
This library is the successor to highlighting-kate, which had some problems that were difficult to resolve given its architecture.
In highlighting-kate, the XML syntax descriptions were converted into individual parsec parsers, which were then compiled. This made it difficult to handle IncludeRules properly without circular imports. There was also no way to load a syntax description dynamically.
Skylighting, by contrast, parses the XML syntax descriptions
into Haskell data structures, which are then interpreted by
a “tokenize” function. IncludeRules can now be handled
properly, and users can add new syntax descriptions
dynamically. It is also now possible to convert .theme
files
directly into styles.
Skylighting is also faster than highlighting-kate, by a factor of 3 in some cases.
Installing
To install the latest release from Hackage, do
stack install skylighting
or
cabal install skylighting
If you want the command-line tool, set the executable
flag
using --flag "skylighting:executable"
in stack or
-fexecutable
in cabal.
The release tarball for the skylighting
package includes generated
files not present in this repository. Building from this repository is
a two-step process. In the first step we build the skylighting-core
package, which provides a program, skylighting-extract
, which reads
XML syntax highlighting definitions from the xml
directory and writes
Haskell source files. In the second we actually build the skylighting
package.
Using cabal:
cd skylighting-core
cabal install -fexecutable
cd ../skylighting
skylighting-extract ../skylighting-core/xml/*.xml
cabal install -fexecutable
Using stack:
stack install --flag skylighting-core:executable skylighting-core
cd skylighting
skylighting-extract ../skylighting-core/xml/*.xml
cd ..
stack install --flag skylighting:executable
Command-line tool
A command-line executable, skylighting
, is installed if
the executable
cabal flag is set in building.
For help, skylighting --help
.
Adding new syntaxes
To compile with additional syntaxes, simply add the syntax definition
(XML) file to the xml
directory of the skylighting-core
package and
repeat the bootstrap build described above.
Note that both the library and the executable can dynamically load
syntax definitions, so you may not need to compile them in. If you
prefer this approach, you can use the skylighting-core
package
directly which provides the XML files without the generated code
produced by the bootstrap process described above.
License
The skylighting
package is licensed under the GPL because some of the
XML syntax descriptions from which its tokenizers are generated are
GPL-licensed. However, the skylighting-core
package, which provides
the core types and functions of this project is licensed under the BSD3
license and bundles the GPL-licensed XML files separately.
References
Kate syntax highlighting documentation: https://docs.kde.org/stable5/en/applications/katepart/highlight.html
Kate highlighting definitions: https://github.com/KDE/syntax-highlighting/tree/master/data/syntax
Changes
Revision history for skylighting
0.7.7 – 2019-02-08
- Depend on skylighting-core 0.7.7.x.
0.7.6 – 2019-02-08
- Depend on skylighting-core 0.7.6.x.
0.7.5 – 2018-12-01
- Depend on skylighting-core 0.7.5.x.
- Add J, sml, typescript.
0.7.4 – 2018-10-08
- Depend on skylighting-core 0.7.4.x.
0.7.3 – 2018-08-27
- Add Sylighting.Syntax.Default and ‘default’ language. This just highlights everything as normal text.
- Require base >= 4.8. Drop support for ghc 7.8.
0.7.2 – 2018-06-08
- Require skylighting-core >= 0.7.2.
0.7.1 – 2018-03-15
- Require skylighting-core >= 0.7.1.
0.7.0.2 – 2018-03-06
- Require skylighting-core >= 0.7.0.2.
0.7.0.1 – 2018-03-03
- Updated changelog.md.
0.7 – 2018-03-03
-
Split skylighting into skylighting-core (BSD3) and skylighting (GPL) (#37, Jonathan Daugherty). skylighting-core (BSD3 licensed) includes the core library functionality and the source of the XML syntax, but not the compiled Syntax modules, and skylighting (GPL2 licensed) includes Syntax modules compiled from the XML syntax files.
-
We discontinue support for GHC before 7.10 (because we need to support the reexport-modules Cabal directive.)
-
Print message about loaded syntax defs to stderr (#36).
-
New module Skylighting.Loader, exporting
loadSyntaxFromFile
andloadSyntaxesFromDir
(Jonathan Daugherty). -
Adjust tests to load xml definitions dynamically.
0.6 – 2018-01-18
-
Add ANSI color output (Alexander Ronald Altman, #22).
- New package dependencies:
ansi-terminal
andcolour
. - New module:
Skylighting.Format.ANSI
, exportingformatANSI
, (also reexported from the moduleSkylighting
). - In
Skylighting.Types
, new typesANSIColorLevel
andXterm256ColorCode
, and a new fieldansiColorLevel
inFormatOptions
. - Main
skylighting
executable now supports ANSI output (useansi
as argument to--format
), which is now the default output format. A new flag--color-level
has been added to select the number of colors (with options16
,256
,true
, and the defaultauto
).
- New package dependencies:
-
Reword error for missing contexts in IncludeRules.
-
Use ordNub instead of nub for missingIncludes.
0.5.1 – 2018-01-07
-
Fixed tokenizer exceptions (#31). Previously
throwError
did not actually generateLeft
results intokenize
, because of the way the MonadPlus instance worked. Rewrote the TokenizerM monad from scratch so that exceptions would not be skipped. -
Improved error message about undefined contexts.
-
Makefile: use cabal for bootstrap target.
-
README: remove stack bootstrap instructions. You can no longer bootstrap with stack, because it insists on building ALL the executables the first time, and of course that can’t be done with this library.
-
Use quickcheck for fuzz test.
0.5.0.1 – 2017-12-18
-
Small improvements to fuzz tests in test suite. We now ensure that we print the random text on test failure. Also, we now run the test with many smaller samples rather than one big one.
-
Add aeson lower bound (because of toEncoding) (#28).
0.5 – 2017-12-10
-
Fix line spacing and overflowing content in generated HTML (David Baynard, #25, see jgm/pandoc#4128).
- Fix empty line height, explicitly
- Ensure long lines scroll on screen
- Only apply colour to the outer div
- Don’t reset line number colour
- Fix borders on empty code lines
- Collapse divs correctly.
-
Changes to Style types and JSON instances. Previously we could not successfully round-trip through JSON.
tokenStyles
is now a map rather than an association list.- We now use
line-number-color
instead ofline-numbers
at the top level in the JSON instances, falling back toline-numbers
ineditor-colors
, for KDE theme compatibility. - We use
line-number-background-color
at the top level, falling back to the text background color. - We use
text-color
at the top level, falling back to thetext-color
of theNormal
token style if it exists, for KDE compatibility. - We use
background-color
at the top level, falling back to thebackground-color
ineditor-colors
, for KDE compatibility. - A round-trip JSON test has been added.
0.4.4.1 – 2017-11-27
- HTML formatting: fix color, bgcolor when numbering enabled. Previously, the code got the color and background color the numbers were supposed to get. See jgm/pandoc #4103.
- Updated syntax descriptions for bash, clojure, commonlisp, diff, dockerfile, doxygen, doxygenlua, fsharp, hamlet, haskell, haxe, java, javascript, julia, latex, literate-curry, literate-haskell, makefile, mediawiki, monobasic, ocaml, prolog, r, relaxng, scala, sci, sql-mysql, sql-postgresql, sql, xslt.
- test program: use –accept instead of –regen.
0.4.4 – 2017-11-21
- HTML formatter: always use an a element (rather than a div) for source lines. The divs were invalid, because code elements must contain phrasing content. Previously we used a elements when line anchors were called for, but there is no clear reason not to use them always.
- skylighting binary: add doctype to generated HTML.
0.4.3.2 – 2017-11-04
- Fixed regression in
data-line-number
attributes in HTML output. We should not include thelineIdPrefix
in front of this.
0.4.3.1 – 2017-11-03
- Fixed typo in css (Artymort).
- Fixed travis build times.
0.4.3 — 2017-11-02
FormatOptions
: addedlineIdPrefix
(jgm/pandoc#4031). This is needed because a web page may contain several code samples, and we need to make sure that the ids on lines are unique.- HTML formatter: use
lineIdPrefix
in ids for lines. - HTML formatter: Don’t put href attributes on divs.
0.4.2 — 2017-10-26
- HTML output: remove outer div. This prevented margin collapsing between the pre and surrounding block elements, and often gave us excess white space around code blocks. See jgm/pandoc#3996.
0.4.1 — 2017-09-28
- Updated XML definitions from KDE repository. Changed: actionscript, ada, agda, alert, alert_indent, asn1, awk, bash, boo, c, changelog, clojure, cmake, coldfusion, commonlisp, cpp, cs, css, curry, d, dockerfile, dot, doxygen, doxygenlua, eiffel, email, erlang, fortran, fsharp, gcc, haskell, haxe, isocpp, java, javascript, jsp, julia, lilypond, lua, m4, makefile, matlab, maxima, mips, modelines, modula-2, monobasic, objectivec, objectivecpp, ocaml, octave, opencl, pascal, perl, php, pike, postscript, prolog, purebasic, python, r, relaxng, relaxngcompact, rest, rhtml, ruby, rust, scala, scheme, sci, sql-mysql, sql-postgresql, sql, tcl, tcsh, verilog, vhdl, xml, xslt, xul, zsh.
- Added support for powershell, using definition from KDE repository.
- Stop terminating long builds on TravisCI (Kyle Ondy).
0.4 — 2017-09-15
- Removed ToJSON/FromJSON instances for KeywordAttr, WordSet, Matcher, Context, ContextSwitch, Rule, Syntax (added in 0.3.5). Creating these increased the memory requirements for compiling skylighting to a degree not justified by the usefulness of these instances.
0.3.5 — 2017-09-14
- Added ToJSON/FromJSON instances for all basic types.
- Added some background colors to ‘kate’ style, matching default.theme, and made FromJSON for Style sensitive to background-color.
0.3.4.1 – 2017-09-09
- HTML formatting: do not use
div
elements for source lines in inline output.
0.3.4 – 2017-09-09
-
HTML formatting changes (David Baynard). Note that these changes may require changes in hard-coded CSS that is used with skylighting’s HTML output.
- Wrap lines of source code in a
div
withdisplay
set toinline-block
. Thediv
s make per-line processing easier. They cannot be set asdisplay: block
as that introduces extra new lines when copying and pasting. - Render line numbers in html using CSS pseudo elements rather than
a table. The line numbers are always produced, as
data-line-number
attributes, and css to display them as::before
elements are always produced. The option to switch on line numbering only toggles a class; this means it is possible to toggle line numbering without re-running skylighting. - If the
linkAnchors
option is set, wrap with ana
element rather than adiv
, set so that clicking the line number (and only the line number) will jump to that line. - Code wraps by default when html is printed, with wrapped lines indented.
- Wrap lines of source code in a
0.3.3.1 – 2017-06-26
- Updated xml syntax definitions and clojure test.
- Improved ‘make update-xml’ target.
- Updated version bound for criterion.
0.3.3 – 2017-04-22
- Revert change from 0.3.2; now entities from a DTD are resolved if the DTD element is included. This is needed for agda.xml and some others. If you want to use a custom xml definition without needing language.dtd, just remove the reference to language.dtd in your xml file.
- Added
case
keyword for ats (Hanwen (Steinway) Wu).
0.3.2 – 2017-04-01
- Parse xml definitions without resolving entities from language.dtd. This allows xml definitions to be used even when language.dtd isn’t available. Existing definitions don’t rely on the two or three entities definied in language.dtd, so this is harmless.
- Small optimizations.
0.3.1 – 2017-02-28
- Use handwritten parser for Float instead of regex. Fixes jgm/highlighting-kate#57 (again).
- Added float parsing tests.
- Use parsec parsers, not regex, for CStringChar and CChar.
- Rewrote Int, Hex, Oct, CStringChar, CCHar parsers with parsec instead of regex. This speeds things up.
- Don’t include leading + in Int, Hex, Oct parsers. That gives
bad results for things like
i+1
.
0.3 – 2017-02-19
- Store literal binary-encoded representations of the Syntax structure in Skylighting.Syntax.*.hs, instead of the structure itself. This works around a problem ghc has compiling modules with large structures. This indirect method produces faster compile times and avoids massive memory usage by ghc (especially in profiling builds). For background see http://stackoverflow.com/questions/16348340/. Closes #7.
- Types: Internals of ‘WordSet’ are no longer exposed.
0.2 – 2017-02-19
- Restore Data, Typeable, Generic instances for basic types. To do this, we remove reCompiled field from RE (API change). Instead, we have the tokenizer compile regexes the first time it encounters them, memoizing the results. Performance is not significantly worse.
- Skylighting.Syntax.*: use string representation of the Syntax, which is then ‘read’, rather than including the code for the data structure directly (#7). This indirect method produces faster compile times and avoids massive memory usage by ghc (especially in profiling builds). For background see http://stackoverflow.com/questions/16348340/compiling-very-large-constants-with-ghc
- Use -fprof-auto-exported for profiling builds.
- Added benchmark for xml syntax definition parsing.
- Patched perl.xml (improperly escaped regex) (#8). This fixes a perl regex compilation error for regex patterns with backslash as delimiter, e.g. m\a.
- Add a flag to build with system pcre (Felix Yan).
0.1.1.5 – 2017-02-03
- Avoid depending on a PrintfArg instance for Text (#5). This isn’t provided in some older versions of the text library. This change should allow the package to build on older platforms.
0.1.1.4 – 2017-01-31
-
Properly escape characters in subDynamic. This fixes a problem that caused failures with
echo -e "s\0b\0c" | skylighting -s perl
-
More efficient rewrite of char escaping for regexes.
0.1.1.3 – 2017-01-29
- Increase test timeout to 6s.
- Avoid double {{}} in latex macros.
- Fixed problem compiling Format.LaTeX on older ghc versions (7.8.3) that can’t take a Text as PrintfArg.
0.1.1.2 – 2017-01-28
- Added much more extensive testing to ensure that tokenizers don’t hang or drop input.
- Fixed some issues with line-end and fallthrough context handling.
- Fixed a bug in WordDetect handling which caused it to drop input (#2).
0.1.1.1 – 2017-01-21
- Optimized. Speed is now comparable to highlighting-kate and often better.
- Added benchmarks.
0.1.1 – 2017-01-19
- Added breezeDark style, from breeze-dark.theme.
- Fixed performance bug in regex application (#1). This gives a significant speedup, especially for inputs with long lines.
0.1.0.1 – 2016-12-23
- Fixed bug in LaTeX renderer (backslash in wrong position).
0.1.0.0 – 2016-12-23
- Initial release