• Download jParser 1.0.0 (recommended)
  
• Download jParser devel package (Full source and build scripts)
• See the library examples running at timwhitlock.info/jparser

The library has been split in two:

1. jTokenizer – A JavaScript tokenizer designed to mimic the PHP tokenizer.
2. jParser - The fully blown JavaScript syntactical parser which generates a parse tree.

The reason for the split is that for most purposes where you think you need a parser, you in fact just need a tokenizer. The tokenizer library is about 15KB, whereas the parser is over 700KB (minified), so you can see why you might not want to include it unnecessarily.

The library files jparser.php and jtokenizer.php are self-contained, minified files for production use. If you wish to inspect or modify the code you will need to download the devel package. This package provides a build script which collates the libraries into their distributable files.

jTokenizer

Possible uses for the tokenizer include code highlighting and simple manipulation of JavaScript source code.

The main function you will want to use is j_token_get_all which behaves the same as the PHP token_get_all function with the addition of a column number as well as a line number. Additionally there is the j_token_name as per the PHP token_name function.

jParser

This is a full, syntactical parser. On its own it simply generates a parse tree which can be traversed and manipulated. There is no proper documentation on this yet, but take a look at the node classes in the devel package if you are serious about doing something useful with this parser.

Some other notes in no particular order

The full parser uses a lot of juice. I recommend giving PHP loads of memory, and be careful what you throw at it if you’re going to run it on a production server.

A parser is not an interpreter or a JavaScript engine. If you want to develop such a thing in PHP you might be insane, but it could be done with this parser as a base.

The JParser parse tree is purposefully not a full tree, it collapses redundant nodes to save memory. If you want to see a full tree then take a look at the JParserRaw class. (devel package required)

Splitting the parsing process into two parts (tokenize/parse) is probably not the most efficient and probably uses more memory than it would another way. However, I figured it would be neat to mimic the PHP tokenizer functionality so that parsers could be built that take a stream of PHP tokens.

↓ JavaScript grammar file for jParser

This file is in a (probably non-standard) variant of BNF notation. I’m not generating the tables with a tool like ANTLR primarily because I don’t write C. I should also point out that I don’t come from a formalized computer science background, so don’t expect this to be 100% conventional.

I’ve developed a native PHP parse table generator that in turn uses a parser (based on itself) to parse this BNF grammar into a table. If you understand grammar files like this you’ll notice something a bit odd – The terminal symbols don’t go right down to individual characters, the grammar expects the source code to already have been tokenized into significant chunks, such as J_NUMBER representing an already identified numeric token. This was done deliberately to be compatibile with PHP’s own Tokenizer functions. The underlying parser framework was designed such that PHP token based grammars could also be developed.

I’ve had to disappoint so far, because it’s not currently available for download. My web analytics also tells me that a lot of people are finding this blog by Googling “php javascript parser“. There’s clearly a need out there, so I’d better do something about it.

I know why I want a JavaScript parser, but what kind of things might you need it for? What API features/functions would you like to see? Please post a comment and let me know. Watch this space for a release, and in the mean time here’s some food for thought …

Do you really need a full parser?

Perhaps all you really need is a tokenizer. Suppose you want to highlight JavaScript code – you don’t need a full parser; all you need to do is break the source code into meaningful chunks (strings, numbers etc..) . How these chunks work together to form syntax is not an issue. I envisage releasing a lighter-weight library containing just the tokenizer (jTokenizer I guess). This will work exactly the same as the PHP tokenizer functions.

Minifying, or obfuscating JavaScript

If you’re after a minifier, there are lighter-weight PHP options available. However, I’ve released a command line version of my JavaScript minifier and obfuscator which is quite handy. It’s more powerful than most JavaScript minifiers, because it supports automatic semicolon insertion. This means it can reduce any code to a single line.