diff --git a/docs/develop/support/json/json.md b/docs/develop/support/json/json.md new file mode 100644 index 0000000000..4b9dfe4b5e --- /dev/null +++ b/docs/develop/support/json/json.md @@ -0,0 +1,109 @@ +# JSON + +[JSON](http://www.json.org/) is a simple textual description of a data structure. An example of some JSON would be; + +``` +[ "apple", "orange", { "drink": "tonic water", "count" : 123 } ] +``` + +## Parsing + +### Generic In-Memory Model + +For some applications, parsing to an in-memory data structure is ideal. In such cases, the ```BJson``` class provides static methods for parsing a block of JSON data into a ```BMessage``` object. + +#### BMessage Structure + +The ```BMessage``` class has the ability to carry a collection of key-value pairs. In the case of a JSON object type, the key-value pairs correlate to the JSON object. In the case of a JSON array type, the key-value pairs are the index of the elements in the JSON array represented as strings. + +For example, the following JSON array... + +``` +[ "a", "b", "c" ] +``` + +...would be represented by the following ```BMessage```; + +|Key|Value| +|---|---| +|"0"|"a"| +|"1"|"b"| +|"2"|"c"| + +A JSON object that, in its entirety, consists of a non-collection type such as a simple string or a boolean is not able to be represented by a ```BMessage```; at the top level there must be an array or an object. + +### Streaming + +Streaming is useful in many situations; + +* where handling the parsed data is easier to undertake as a stream of events +* where the quantity of input or output data could be non-trivial and holding that quantity of material in memory is undesirable +* where being able to start processing a stream of data before the entire payload has arrived is desirable + +This architecture is sometimes known as an event-based parser or a "SAX" parser. + +The ```BJson``` class provides a static method that accepts a stream of JSON data in the form of a ```BDataIO``` and a ```BJsonEventListener``` sub-class. As each token is processed from the stream, it will provided to the listener. The listener must implement three callback methods to handle the JSON parsing; + +|Method|Description| +|---|---| +|Handle(..)|Provides JSON events to the listener| +|HandleError(..)|Signals parse or processing errors to the listener| +|Complete(..)|Informs the listener that parsing has completed| + +Events are embodied in instances of the ```BJsonEvent``` class and each of these has a type. Sample example types are; + +* B_JSON_STRING +* B_JSON_OBJECT_START +* B_JSON_TRUE + +In this way, the listener is able to interpret the incoming stream of data as JSON and handle it in some way. + +The following JSON... + +``` +{"color": "red", "alpha": 0.6} +``` + +Would yield the following stream of events; + +|Event Type|Event Data| +|---|---| +|B_JSON_OBJECT_START|-| +|B_JSON_OBJECT_NAME|"color"| +|B_JSON_STRING|"red"| +|B_JSON_OBJECT_NAME|"alpha"| +|B_JSON_NUMBER|0.6| +|B_JSON_OBJECT_END|-| + +#### Number Handling + +The JSON number literal format does not specify a numeric type such as ```int32``` or ```double```. To cope with the widest range of possibilities, the ```B_JSON_NUMBER``` event type captures the content as a string and then the ```BJsonEvent``` object is able to provide the original string for specific handling as well as convenient accessors for parsing to ```double``` or ```int64``` types. This provides a high level of flexibility for the client. + +#### Stacked Listeners + +One implementation approach for the listener used to read a data-transfer-object (DTO) is to create "sub-listeners" that mirror the structure of the JSON. + +In the following example, a nested data structure is being parsed. + +![Stacked Listeners](stacked-listeners.svg) + +A primary-listener is employed called ```ColorGradientsListener```. The primary-listener accepts JSON parse events and will relay them to a sub-listener. The sub-listener is implemented to specifically deal with one tier of the inbound data. The sub-listeners are structured in a stack where the sub-listener at the head of the stack has a pointer to it's parent. The primary-listener maintains a pointer to the current head of the stack and will direct events to that sub-listener. + +In response to events, the sub-listener can take-up the data, pop itself from the stack or push additional sub-listeners from the stack. + +The same approach has been used in the following classes in a more generic manner; + +* BJsonTextWriter +* BJsonMessageWriter + +The intention with this approach is that the structure of the event handling code in the sub-listeners mirrors that of the data-structure being parsed. Hopefully this makes creating the filling of a specific data-model easier even when very specific behaviours are required. + + From a schema of the data structure it is _probably_ also possible to create these sub-listeners and in this way automatically generate the C++ parse code as event listeners. + +## Writing + +In order to render a data-structure as JSON data, the opposite occurs; events are emitted by the client software into a class ```BJsonTextWriter```. This class supports public methods such as ```WriteFalse()```, ```WriteObjectStart()``` and ```WriteString(...)``` that control the outbound JSON stream. + +### End to End + +Because ```BJsonTextWriter``` is accepting JSON parse events, it is also a ```JsonEventListener``` and so can be used as a listener with the stream parsing; producing JSON output from JSON input. The output will however not include inbound whitespace because whitespace is not grammatically significant in JSON. \ No newline at end of file diff --git a/docs/develop/support/json/stacked-listeners.svg b/docs/develop/support/json/stacked-listeners.svg new file mode 100644 index 0000000000..b8f6329a5e --- /dev/null +++ b/docs/develop/support/json/stacked-listeners.svg @@ -0,0 +1,381 @@ + + + + + + + + + + + + image/svg+xml + + + + + + + + + + + + [ { "code": "SUNCOLS", "name: "Sunset", "colors": [ { "code": "REDS", "color": "#FF0000" } ] }, { "code": "WATER", "name": "Water Colors", "colors": [ { "code": "BLUE", "color": "#0000FF" }, { "code": "WHITE", "color": "#FFFFFF" } ] }] + + + ColorGradientsSubListener + + ColorGradientSubListener + + ColorSubListener + + + + + + ColorGradientsListener + + + diff --git a/headers/private/shared/Json.h b/headers/private/shared/Json.h index edb8e453b4..73746f2177 100644 --- a/headers/private/shared/Json.h +++ b/headers/private/shared/Json.h @@ -1,32 +1,64 @@ /* + * Copyright 2017, Andrew Lindesay * Copyright 2014, Augustin Cavalier (waddlesplash) * Distributed under the terms of the MIT License. */ #ifndef _JSON_H #define _JSON_H + +#include "JsonEventListener.h" + #include #include + namespace BPrivate { +class JsonParseContext; + class BJson { -public: - enum JsonObjectType { - JSON_TYPE_MAP = '_JTM', - JSON_TYPE_ARRAY = '_JTA' - }; public: static status_t Parse(const char* JSON, BMessage& message); static status_t Parse(const BString& JSON, BMessage& message); + static void Parse(BDataIO* data, + BJsonEventListener* listener); private: - static void _Parse(const BString& JSON, BMessage& message); - static BString _ParseString(const BString& JSON, int32& pos); - static double _ParseNumber(const BString& JSON, int32& pos); - static bool _ParseConstant(const BString& JSON, int32& pos, - const char* constant); + static bool NextChar(JsonParseContext& jsonParseContext, + char* c); + static bool NextNonWhitespaceChar( + JsonParseContext& jsonParseContext, + char* c); + + static bool ParseAny(JsonParseContext& jsonParseContext); + static bool ParseObjectNameValuePair( + JsonParseContext& jsonParseContext); + static bool ParseObject(JsonParseContext& jsonParseContext); + static bool ParseArray(JsonParseContext& jsonParseContext); + static bool ParseEscapeUnicodeSequence( + JsonParseContext& jsonParseContext, + BString& stringResult); + static bool ParseStringEscapeSequence( + JsonParseContext& jsonParseContext, + BString& stringResult); + static bool ParseString(JsonParseContext& jsonParseContext, + json_event_type eventType); + static bool ParseExpectedVerbatimStringAndRaiseEvent( + JsonParseContext& jsonParseContext, + const char* expectedString, + size_t expectedStringLength, + char leadingChar, + json_event_type jsonEventType); + static bool ParseExpectedVerbatimString( + JsonParseContext& jsonParseContext, + const char* expectedString, + size_t expectedStringLength, + char leadingChar); + + static bool IsValidNumber(BString& number); + static bool ParseNumber(JsonParseContext& jsonParseContext); }; } // namespace BPrivate diff --git a/headers/private/shared/JsonEvent.h b/headers/private/shared/JsonEvent.h new file mode 100644 index 0000000000..039899caa4 --- /dev/null +++ b/headers/private/shared/JsonEvent.h @@ -0,0 +1,59 @@ +/* + * Copyright 2017, Andrew Lindesay + * Distributed under the terms of the MIT License. + */ +#ifndef _JSON_EVENT_H +#define _JSON_EVENT_H + + +#include + + +/*! This enumeration defines the types of events that may arise when parsing a + stream of JSON data. +*/ + +typedef enum json_event_type { + B_JSON_NUMBER = 1, + B_JSON_STRING = 2, + B_JSON_TRUE = 3, + B_JSON_FALSE = 4, + B_JSON_NULL = 5, + B_JSON_OBJECT_START = 6, + B_JSON_OBJECT_END = 7, + B_JSON_OBJECT_NAME = 8, // aka field + B_JSON_ARRAY_START = 9, + B_JSON_ARRAY_END = 10 +} json_event_type; + + +namespace BPrivate { + +class BJsonEvent { +public: + BJsonEvent(json_event_type eventType, + const char* content); + BJsonEvent(const char* content); + BJsonEvent(double content); + BJsonEvent(int64 content); + BJsonEvent(json_event_type eventType); + ~BJsonEvent(); + + json_event_type EventType() const; + + const char* Content() const; + double ContentDouble() const; + int64 ContentInteger() const; + +private: + + json_event_type fEventType; + const char* fContent; + char* fOwnedContent; +}; + +} // namespace BPrivate + +using BPrivate::BJsonEvent; + +#endif // _JSON_EVENT_H diff --git a/headers/private/shared/JsonEventListener.h b/headers/private/shared/JsonEventListener.h new file mode 100644 index 0000000000..bc629c3ced --- /dev/null +++ b/headers/private/shared/JsonEventListener.h @@ -0,0 +1,37 @@ +/* + * Copyright 2017, Andrew Lindesay + * Distributed under the terms of the MIT License. + */ +#ifndef _JSON_EVENT_LISTENER_H +#define _JSON_EVENT_LISTENER_H + + +#include "JsonEvent.h" + +#include + + +/*! This constant line number can be used in raising an error where the + client raising the error does not know on which line the error arose. +*/ + +#define JSON_EVENT_LISTENER_ANY_LINE -1 + +namespace BPrivate { + +class BJsonEventListener { +public: + BJsonEventListener(); + virtual ~BJsonEventListener(); + + virtual bool Handle(const BJsonEvent& event) = 0; + virtual void HandleError(status_t status, int32 line, + const char* message) = 0; + virtual void Complete() = 0; +}; + +} // namespace BPrivate + +using BPrivate::BJsonEventListener; + +#endif // _JSON_EVENT_LISTENER_H diff --git a/headers/private/shared/JsonMessageWriter.h b/headers/private/shared/JsonMessageWriter.h new file mode 100644 index 0000000000..394a8ff61e --- /dev/null +++ b/headers/private/shared/JsonMessageWriter.h @@ -0,0 +1,48 @@ +/* + * Copyright 2017, Andrew Lindesay + * Distributed under the terms of the MIT License. + */ +#ifndef _JSON_MESSAGE_WRITER_H +#define _JSON_MESSAGE_WRITER_H + + +#include "JsonWriter.h" + +#include +#include + + +enum json_message_container_what { + B_JSON_MESSAGE_WHAT_OBJECT = '_JTM', + B_JSON_MESSAGE_WHAT_ARRAY = '_JTA' +}; + + +namespace BPrivate { + +class BStackedMessageEventListener; + +class BJsonMessageWriter : public BJsonWriter { +friend class BStackedMessageEventListener; +public: + BJsonMessageWriter(BMessage& message); + virtual ~BJsonMessageWriter(); + + bool Handle(const BJsonEvent& event); + void Complete(); + +private: + void SetStackedListener( + BStackedMessageEventListener* listener); + + BMessage* fTopLevelMessage; + BStackedMessageEventListener* + fStackedListener; +}; + + +} // namespace BPrivate + +using BPrivate::BJsonMessageWriter; + +#endif // _JSON_MESSAGE_WRITER_H diff --git a/headers/private/shared/JsonTextWriter.h b/headers/private/shared/JsonTextWriter.h new file mode 100644 index 0000000000..a4e57f3d66 --- /dev/null +++ b/headers/private/shared/JsonTextWriter.h @@ -0,0 +1,61 @@ +/* + * Copyright 2017, Andrew Lindesay + * Distributed under the terms of the MIT License. + */ +#ifndef _JSON_STRING_STREAM_WRITER_H +#define _JSON_STRING_STREAM_WRITER_H + + +#include "JsonWriter.h" + +#include +#include + + +namespace BPrivate { + +class BJsonTextWriterStackedEventListener; + +class BJsonTextWriter : public BJsonWriter { +friend class BJsonTextWriterStackedEventListener; +public: + BJsonTextWriter(BDataIO* dataIO); + virtual ~BJsonTextWriter(); + + bool Handle(const BJsonEvent& event); + void Complete(); + +private: + void SetStackedListener( + BJsonTextWriterStackedEventListener* + stackedListener); + + status_t StreamNumberNode(const BJsonEvent& event); + + status_t StreamStringVerbatim(const char* string); + status_t StreamStringVerbatim(const char* string, + off_t offset, size_t length); + + status_t StreamStringEncoded(const char* string); + status_t StreamStringEncoded(const char* string, + off_t offset, size_t length); + + status_t StreamQuotedEncodedString(const char* string); + status_t StreamQuotedEncodedString(const char* string, + off_t offset, size_t length); + + status_t StreamChar(char c); + + BDataIO* fDataIO; + BJsonTextWriterStackedEventListener* + fStackedListener; + + char fUnicodeAssemblyBuffer[7]; + +}; + +} // namespace BPrivate + +using BPrivate::BJsonTextWriter; + +#endif // _JSON_STRING_STREAM_WRITER_H diff --git a/headers/private/shared/JsonWriter.h b/headers/private/shared/JsonWriter.h new file mode 100644 index 0000000000..67ae5114f3 --- /dev/null +++ b/headers/private/shared/JsonWriter.h @@ -0,0 +1,52 @@ +/* + * Copyright 2017, Andrew Lindesay + * Distributed under the terms of the MIT License. + */ +#ifndef _JSON_WRITER_H +#define _JSON_WRITER_H + + +#include "JsonEventListener.h" + +#include +#include + + +namespace BPrivate { + +class BJsonWriter : public BJsonEventListener { +public: + BJsonWriter(); + virtual ~BJsonWriter(); + + void HandleError(status_t status, int32 line, + const char* message); + status_t ErrorStatus(); + + status_t WriteBoolean(bool value); + status_t WriteTrue(); + status_t WriteFalse(); + status_t WriteNull(); + + status_t WriteInteger(int64 value); + status_t WriteDouble(double value); + + status_t WriteString(const char* value); + + status_t WriteObjectStart(); + status_t WriteObjectName(const char* value); + status_t WriteObjectEnd(); + + status_t WriteArrayStart(); + status_t WriteArrayEnd(); + +protected: + status_t fErrorStatus; + +}; + +} // namespace BPrivate + +using BPrivate::BJsonWriter; + +#endif // _JSON_WRITER_H