jsonpickle Documentation¶
jsonpickle
is a Python library for
serialization and deserialization of complex Python objects to and from
JSON. The standard Python libraries for encoding Python into JSON, such as
the stdlib’s json, simplejson, and demjson, can only handle Python
primitives that have a direct JSON equivalent (e.g. dicts, lists, strings,
ints, etc.). jsonpickle builds on top of these libraries and allows more
complex data structures to be serialized to JSON. jsonpickle is highly
configurable and extendable–allowing the user to choose the JSON backend
and add additional backends.
Contents
jsonpickle Usage¶
Note
Please see the note in the jsonpickle – High Level API when serializing dictionaries that contain non-string dictionary keys.
Python library for serializing any arbitrary object graph into JSON.
Warning
jsonpickle can execute arbitrary Python code. Do not load jsonpickles from untrusted / unauthenticated sources.
jsonpickle can take almost any Python object and turn the object into JSON. Additionally, it can reconstitute the object back into Python.
The object must be accessible globally via a module and must inherit from object (AKA new-style classes).
Create an object:
class Thing(object):
def __init__(self, name):
self.name = name
obj = Thing('Awesome')
Use jsonpickle to transform the object into a JSON string:
import jsonpickle
frozen = jsonpickle.encode(obj)
Use jsonpickle to recreate a Python object from a JSON string:
thawed = jsonpickle.decode(frozen)
The new object has the same type and data, but essentially is now a copy of the original.
assert obj.name == thawed.name
If you will never need to load (regenerate the Python class from JSON), you can pass in the keyword unpicklable=False to prevent extra information from being added to JSON:
oneway = jsonpickle.encode(obj, unpicklable=False)
result = jsonpickle.decode(oneway)
assert obj.name == result['name'] == 'Awesome'
Download & Install¶
The easiest way to get jsonpickle is via PyPi with pip:
$ pip install -U jsonpickle
jsonpickle has no required dependencies (it uses the standard library’s
json
module by default).
You can also download or checkout the latest code and install from source:
$ python setup.py install
API Reference¶
jsonpickle API¶
Contents
-
jsonpickle
– High Level APIjsonpickle.handlers
– Custom Serialization Handlers
-
jsonpickle.pickler
– Python to JSON-compatible dictjsonpickle.unpickler
– JSON-compatible dict to Pythonjsonpickle.backend
– JSON Backend Managementjsonpickle.util
– Helper functions
jsonpickle
– High Level API¶
Note
For performance and compatibility reasons, jsonpickle does not preserve non-string dictionary keys by default. This results in a simpler, but sometimes lossy, JSON representation.
Specify keys=True
when encoding and decoding to preserve integers,
tuples, and other non-strings types as dictionary keys.
-
jsonpickle.
encode
(value, unpicklable=True, make_refs=True, keys=False, max_depth=None, reset=True, backend=None, warn=False, context=None, max_iter=None, use_decimal=False, numeric_keys=False, use_base85=False, fail_safe=None, indent=None, separators=None)¶ Return a JSON formatted representation of value, a Python object.
- Parameters
unpicklable – If set to False then the output will not contain the information necessary to turn the JSON data back into Python objects, but a simpler JSON stream is produced.
max_depth – If set to a non-negative integer then jsonpickle will not recurse deeper than ‘max_depth’ steps into the object. Anything deeper than ‘max_depth’ is represented using a Python repr() of the object.
make_refs – If set to False jsonpickle’s referencing support is disabled. Objects that are id()-identical won’t be preserved across encode()/decode(), but the resulting JSON stream will be conceptually simpler. jsonpickle detects cyclical objects and will break the cycle by calling repr() instead of recursing when make_refs is set False.
keys – If set to True then jsonpickle will encode non-string dictionary keys instead of coercing them into strings via repr(). This is typically what you want if you need to support Integer or objects as dictionary keys.
numeric_keys – Only use this option if the backend supports integer dict keys natively. This flag tells jsonpickle to leave numeric keys as-is rather than conforming them to json-friendly strings. Using
keys=True
is the typical solution for integer keys, so only use this if you have a specific use case where you want to allow the backend to handle serialization of numeric dict keys.warn – If set to True then jsonpickle will warn when it returns None for an object which it cannot pickle (e.g. file descriptors).
max_iter – If set to a non-negative integer then jsonpickle will consume at most max_iter items when pickling iterators.
use_decimal –
If set to True jsonpickle will allow Decimal instances to pass-through, with the assumption that the simplejson backend will be used in use_decimal mode. In order to use this mode you will need to configure simplejson:
jsonpickle.set_encoder_options('simplejson', use_decimal=True, sort_keys=True) jsonpickle.set_decoder_options('simplejson', use_decimal=True) jsonpickle.set_preferred_backend('simplejson')
NOTE: A side-effect of the above settings is that float values will be converted to Decimal when converting to json.
use_base85 – If possible, use base85 to encode binary data. Base85 bloats binary data by 1/4 as opposed to base64, which expands it by 1/3. This argument is ignored on Python 2 because it doesn’t support it.
fail_safe – If set to a function exceptions are ignored when pickling and if a exception happens the function is called and the return value is used as the value for the object that caused the error
indent – When indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines.
None
is the most compact representation. Since the default item separator is(', ', ': ')
, the output might include trailing whitespace whenindent
is specified. You can useseparators=(',', ': ')
to avoid this. This value is passed directly to the active JSON backend library and not used by jsonpickle directly.separators – If
separators
is an(item_separator, dict_separator)
tuple then it will be used instead of the default(', ', ': ')
separators.(',', ':')
is the most compact JSON representation. This value is passed directly to the active JSON backend library and not used by jsonpickle directly.
>>> encode('my string') == '"my string"' True >>> encode(36) == '36' True >>> encode({'foo': True}) == '{"foo": true}' True >>> encode({'foo': [1, 2, [3, 4]]}, max_depth=1) '{"foo": "[1, 2, [3, 4]]"}'
-
jsonpickle.
decode
(string, backend=None, context=None, keys=False, reset=True, safe=False, classes=None)¶ Convert a JSON string into a Python object.
The keyword argument ‘keys’ defaults to False. If set to True then jsonpickle will decode non-string dictionary keys into python objects via the jsonpickle protocol.
The keyword argument ‘classes’ defaults to None. If set to a single class, or a sequence (list, set, tuple) of classes, then the classes will be made available when constructing objects. This can be used to give jsonpickle access to local classes that are not available through the global module import scope.
>>> decode('"my string"') == 'my string' True >>> decode('36') 36
Choosing and Loading Backends¶
jsonpickle allows the user to specify what JSON backend to use
when encoding and decoding. By default, jsonpickle will try to use, in
the following order: simplejson
, json
, and demjson
.
The prefered backend can be set via jsonpickle.set_preferred_backend()
.
Additional JSON backends can be used via jsonpickle.load_backend()
.
For example, users of Django can use the version of simplejson that is bundled in Django:
jsonpickle.load_backend('django.util.simplejson', 'dumps', 'loads', ValueError))
jsonpickle.set_preferred_backend('django.util.simplejson')
Supported backends:
Experimental backends:
-
jsonpickle.
set_preferred_backend
(name)¶ Set the preferred json backend.
If a preferred backend is set then jsonpickle tries to use it before any other backend.
For example:
set_preferred_backend('simplejson')
If the backend is not one of the built-in jsonpickle backends (json/simplejson, or demjson) then you must load the backend prior to calling set_preferred_backend.
AssertionError is raised if the backend has not been loaded.
-
jsonpickle.
load_backend
(name, dumps='dumps', loads='loads', loads_exc=<class 'ValueError'>)¶ Load a JSON backend by name.
This method loads a backend and sets up references to that backend’s loads/dumps functions and exception classes.
- Parameters
dumps – is the name of the backend’s encode method. The method should take an object and return a string. Defaults to ‘dumps’.
loads – names the backend’s method for the reverse operation – returning a Python object from a string.
loads_exc – can be either the name of the exception class used to denote decoding errors, or it can be a direct reference to the appropriate exception class itself. If it is a name, then the assumption is that an exception class of that name can be found in the backend module’s namespace.
load – names the backend’s ‘load’ method.
dump – names the backend’s ‘dump’ method.
- Rtype bool
True on success, False if the backend could not be loaded.
-
jsonpickle.
remove_backend
(name)¶ Remove all entries for a particular backend.
-
jsonpickle.
set_encoder_options
(name, *args, **kwargs)¶ Associate encoder-specific options with an encoder.
After calling set_encoder_options, any calls to jsonpickle’s encode method will pass the supplied args and kwargs along to the appropriate backend’s encode method.
For example:
set_encoder_options('simplejson', sort_keys=True, indent=4) set_encoder_options('demjson', compactly=False)
See the appropriate encoder’s documentation for details about the supported arguments and keyword arguments.
-
jsonpickle.
set_decoder_options
(name, *args, **kwargs)¶ Associate decoder-specific options with a decoder.
After calling set_decoder_options, any calls to jsonpickle’s decode method will pass the supplied args and kwargs along to the appropriate backend’s decode method.
For example:
set_decoder_options('simplejson', encoding='utf8', cls=JSONDecoder) set_decoder_options('demjson', strict=True)
See the appropriate decoder’s documentation for details about the supported arguments and keyword arguments.
Customizing JSON output¶
jsonpickle supports the standard pickle
__getstate__ and __setstate__
protocol for representating object instances.
-
object.
__getstate__
()¶ Classes can further influence how their instances are pickled; if the class defines the method
__getstate__()
, it is called and the return state is pickled as the contents for the instance, instead of the contents of the instance’s dictionary. If there is no__getstate__()
method, the instance’s__dict__
is pickled.
-
object.
__setstate__
(state)¶ Upon unpickling, if the class also defines the method
__setstate__()
, it is called with the unpickled state. If there is no__setstate__()
method, the pickled state must be a dictionary and its items are assigned to the new instance’s dictionary. If a class defines both__getstate__()
and__setstate__()
, the state object needn’t be a dictionary and these methods can do what they want.
jsonpickle.handlers
– Custom Serialization Handlers¶
The jsonpickle.handlers module allows plugging in custom serialization handlers at run-time. This feature is useful when jsonpickle is unable to serialize objects that are not under your direct control.
Custom handlers may be created to handle other objects. Each custom handler
must derive from jsonpickle.handlers.BaseHandler
and
implement flatten
and restore
.
A handler can be bound to other types by calling
jsonpickle.handlers.register()
.
-
class
jsonpickle.handlers.
ArrayHandler
(context)¶ Flatten and restore array.array objects
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
BaseHandler
(context)¶ -
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
classmethod
handles
(cls)¶ Register this handler for the given class. Suitable as a decorator, e.g.:
@MyCustomHandler.handles class MyCustomClass: def __reduce__(self): ...
-
restore
(obj)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
CloneFactory
(exemplar)¶ Serialization proxy for collections.defaultdict’s default_factory
-
class
jsonpickle.handlers.
DatetimeHandler
(context)¶ Custom handler for datetime objects
Datetime objects use __reduce__, and they generate binary strings encoding the payload. This handler encodes that payload to reconstruct the object.
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
LockHandler
(context)¶ Serialize threading.Lock objects
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
QueueHandler
(context)¶ Opaquely serializes Queue objects
Queues contains mutex and condition variables which cannot be serialized. Construct a new Queue instance when restoring.
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
RegexHandler
(context)¶ Flatten _sre.SRE_Pattern (compiled regex) objects
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
class
jsonpickle.handlers.
Registry
¶ -
get
(cls_or_name, default=None)¶ - Parameters
cls_or_name – the type or its fully qualified name
default – default value, if a matching handler is not found
Looks up a handler by type reference or its fully qualified name. If a direct match is not found, the search is performed over all handlers registered with base=True.
-
register
(cls, handler=None, base=False)¶ Register the a custom handler for a class
- Parameters
cls – The custom object class to handle
handler – The custom handler class (if None, a decorator wrapper is returned)
base – Indicates whether the handler should be registered for all subclasses
This function can be also used as a decorator by omitting the handler argument:
@jsonpickle.handlers.register(Foo, base=True) class FooHandler(jsonpickle.handlers.BaseHandler): pass
-
unregister
(cls)¶
-
-
class
jsonpickle.handlers.
TextIOHandler
(context)¶ Serialize file descriptors as None because we cannot roundtrip
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore should never get called because flatten() returns None
-
-
class
jsonpickle.handlers.
UUIDHandler
(context)¶ Serialize uuid.UUID objects
-
flatten
(obj, data)¶ Flatten obj into a json-friendly form and write result to data.
-
restore
(data)¶ Restore an object of the registered type from the json-friendly representation obj and return it.
-
-
jsonpickle.handlers.
get
(cls_or_name, default=None)¶ - Parameters
cls_or_name – the type or its fully qualified name
default – default value, if a matching handler is not found
Looks up a handler by type reference or its fully qualified name. If a direct match is not found, the search is performed over all handlers registered with base=True.
-
jsonpickle.handlers.
register
(cls, handler=None, base=False)¶ Register the a custom handler for a class
- Parameters
cls – The custom object class to handle
handler – The custom handler class (if None, a decorator wrapper is returned)
base – Indicates whether the handler should be registered for all subclasses
This function can be also used as a decorator by omitting the handler argument:
@jsonpickle.handlers.register(Foo, base=True) class FooHandler(jsonpickle.handlers.BaseHandler): pass
-
jsonpickle.handlers.
unregister
(cls)¶
Low Level API¶
Typically this low level functionality is not needed by clients.
Note that arguments like safe=True
do not make it safe to load an untrusted
jsonpickle string.
jsonpickle.pickler
– Python to JSON-compatible dict¶
-
class
jsonpickle.pickler.
Pickler
(unpicklable=True, make_refs=True, max_depth=None, backend=None, keys=False, warn=False, max_iter=None, numeric_keys=False, use_decimal=False, use_base85=False, fail_safe=None)¶ -
flatten
(obj, reset=True)¶ Takes an object and returns a JSON-safe representation of it.
Simply returns any of the basic builtin datatypes
>>> p = Pickler() >>> p.flatten('hello world') == 'hello world' True >>> p.flatten(49) 49 >>> p.flatten(350.0) 350.0 >>> p.flatten(True) True >>> p.flatten(False) False >>> r = p.flatten(None) >>> r is None True >>> p.flatten(False) False >>> p.flatten([1, 2, 3, 4]) [1, 2, 3, 4] >>> p.flatten((1,2,))[tags.TUPLE] [1, 2] >>> p.flatten({'key': 'value'}) == {'key': 'value'} True
-
reset
()¶
-
-
jsonpickle.pickler.
encode
(value, unpicklable=True, make_refs=True, keys=False, max_depth=None, reset=True, backend=None, warn=False, context=None, max_iter=None, use_decimal=False, numeric_keys=False, use_base85=False, fail_safe=None, indent=None, separators=None)¶ Return a JSON formatted representation of value, a Python object.
- Parameters
unpicklable – If set to False then the output will not contain the information necessary to turn the JSON data back into Python objects, but a simpler JSON stream is produced.
max_depth – If set to a non-negative integer then jsonpickle will not recurse deeper than ‘max_depth’ steps into the object. Anything deeper than ‘max_depth’ is represented using a Python repr() of the object.
make_refs – If set to False jsonpickle’s referencing support is disabled. Objects that are id()-identical won’t be preserved across encode()/decode(), but the resulting JSON stream will be conceptually simpler. jsonpickle detects cyclical objects and will break the cycle by calling repr() instead of recursing when make_refs is set False.
keys – If set to True then jsonpickle will encode non-string dictionary keys instead of coercing them into strings via repr(). This is typically what you want if you need to support Integer or objects as dictionary keys.
numeric_keys – Only use this option if the backend supports integer dict keys natively. This flag tells jsonpickle to leave numeric keys as-is rather than conforming them to json-friendly strings. Using
keys=True
is the typical solution for integer keys, so only use this if you have a specific use case where you want to allow the backend to handle serialization of numeric dict keys.warn – If set to True then jsonpickle will warn when it returns None for an object which it cannot pickle (e.g. file descriptors).
max_iter – If set to a non-negative integer then jsonpickle will consume at most max_iter items when pickling iterators.
use_decimal –
If set to True jsonpickle will allow Decimal instances to pass-through, with the assumption that the simplejson backend will be used in use_decimal mode. In order to use this mode you will need to configure simplejson:
jsonpickle.set_encoder_options('simplejson', use_decimal=True, sort_keys=True) jsonpickle.set_decoder_options('simplejson', use_decimal=True) jsonpickle.set_preferred_backend('simplejson')
NOTE: A side-effect of the above settings is that float values will be converted to Decimal when converting to json.
use_base85 – If possible, use base85 to encode binary data. Base85 bloats binary data by 1/4 as opposed to base64, which expands it by 1/3. This argument is ignored on Python 2 because it doesn’t support it.
fail_safe – If set to a function exceptions are ignored when pickling and if a exception happens the function is called and the return value is used as the value for the object that caused the error
indent – When indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0 will only insert newlines.
None
is the most compact representation. Since the default item separator is(', ', ': ')
, the output might include trailing whitespace whenindent
is specified. You can useseparators=(',', ': ')
to avoid this. This value is passed directly to the active JSON backend library and not used by jsonpickle directly.separators – If
separators
is an(item_separator, dict_separator)
tuple then it will be used instead of the default(', ', ': ')
separators.(',', ':')
is the most compact JSON representation. This value is passed directly to the active JSON backend library and not used by jsonpickle directly.
>>> encode('my string') == '"my string"' True >>> encode(36) == '36' True >>> encode({'foo': True}) == '{"foo": true}' True >>> encode({'foo': [1, 2, [3, 4]]}, max_depth=1) '{"foo": "[1, 2, [3, 4]]"}'
jsonpickle.unpickler
– JSON-compatible dict to Python¶
-
class
jsonpickle.unpickler.
Unpickler
(backend=None, keys=False, safe=False)¶ -
register_classes
(classes)¶ Register one or more classes
- Parameters
classes – sequence of classes or a single class to register
-
reset
()¶ Resets the object’s internal state.
-
restore
(obj, reset=True, classes=None)¶ Restores a flattened object to its original python state.
Simply returns any of the basic builtin types
>>> u = Unpickler() >>> u.restore('hello world') == 'hello world' True >>> u.restore({'key': 'value'}) == {'key': 'value'} True
-
-
jsonpickle.unpickler.
decode
(string, backend=None, context=None, keys=False, reset=True, safe=False, classes=None)¶ Convert a JSON string into a Python object.
The keyword argument ‘keys’ defaults to False. If set to True then jsonpickle will decode non-string dictionary keys into python objects via the jsonpickle protocol.
The keyword argument ‘classes’ defaults to None. If set to a single class, or a sequence (list, set, tuple) of classes, then the classes will be made available when constructing objects. This can be used to give jsonpickle access to local classes that are not available through the global module import scope.
>>> decode('"my string"') == 'my string' True >>> decode('36') 36
-
jsonpickle.unpickler.
getargs
(obj, classes=None)¶ Return arguments suitable for __new__()
-
jsonpickle.unpickler.
has_tag
(obj, tag)¶ Helper class that tests to see if the obj is a dictionary and contains a particular key/tag.
>>> obj = {'test': 1} >>> has_tag(obj, 'test') True >>> has_tag(obj, 'fail') False
>>> has_tag(42, 'fail') False
-
jsonpickle.unpickler.
loadclass
(module_and_name, classes=None)¶ Loads the module and returns the class.
>>> cls = loadclass('datetime.datetime') >>> cls.__name__ 'datetime'
>>> loadclass('does.not.exist')
>>> loadclass('builtins.int')() 0
-
jsonpickle.unpickler.
loadrepr
(reprstr)¶ Returns an instance of the object from the object’s repr() string. It involves the dynamic specification of code.
>>> obj = loadrepr('datetime/datetime.datetime.now()') >>> obj.__class__.__name__ 'datetime'
-
jsonpickle.unpickler.
make_blank_classic
(cls)¶ Implement the mandated strategy for dealing with classic classes which cannot be instantiated without __getinitargs__ because they take parameters
jsonpickle.backend
– JSON Backend Management¶
-
class
jsonpickle.backend.
JSONBackend
(fallthrough=True)¶ Manages encoding and decoding using various backends.
- It tries these modules in this order:
simplejson, json, demjson
simplejson is a fast and popular backend and is tried first. json comes with Python and is tried second. demjson is the most permissive backend and is tried last.
-
decode
(string)¶ Attempt to decode an object from a JSON string.
This tries the loaded backends in order and passes along the last exception if no backends are able to decode the string.
-
dumps
(obj, indent=None, separators=None)¶ Attempt to encode an object into JSON.
This tries the loaded backends in order and passes along the last exception if no backend is able to encode the object.
-
enable_fallthrough
(enable)¶ Disable jsonpickle’s fallthrough-on-error behavior
By default, jsonpickle tries the next backend when decoding or encoding using a backend fails.
This can make it difficult to force jsonpickle to use a specific backend, and catch errors, because the error will be suppressed and may not be raised by the subsequent backend.
Calling enable_backend(False) will make jsonpickle immediately re-raise any exceptions raised by the backends.
-
encode
(obj, indent=None, separators=None)¶ Attempt to encode an object into JSON.
This tries the loaded backends in order and passes along the last exception if no backend is able to encode the object.
-
load_backend
(name, dumps='dumps', loads='loads', loads_exc=<class 'ValueError'>)¶ Load a JSON backend by name.
This method loads a backend and sets up references to that backend’s loads/dumps functions and exception classes.
- Parameters
dumps – is the name of the backend’s encode method. The method should take an object and return a string. Defaults to ‘dumps’.
loads – names the backend’s method for the reverse operation – returning a Python object from a string.
loads_exc – can be either the name of the exception class used to denote decoding errors, or it can be a direct reference to the appropriate exception class itself. If it is a name, then the assumption is that an exception class of that name can be found in the backend module’s namespace.
load – names the backend’s ‘load’ method.
dump – names the backend’s ‘dump’ method.
- Rtype bool
True on success, False if the backend could not be loaded.
-
loads
(string)¶ Attempt to decode an object from a JSON string.
This tries the loaded backends in order and passes along the last exception if no backends are able to decode the string.
-
remove_backend
(name)¶ Remove all entries for a particular backend.
-
set_decoder_options
(name, *args, **kwargs)¶ Associate decoder-specific options with a decoder.
After calling set_decoder_options, any calls to jsonpickle’s decode method will pass the supplied args and kwargs along to the appropriate backend’s decode method.
For example:
set_decoder_options('simplejson', encoding='utf8', cls=JSONDecoder) set_decoder_options('demjson', strict=True)
See the appropriate decoder’s documentation for details about the supported arguments and keyword arguments.
-
set_encoder_options
(name, *args, **kwargs)¶ Associate encoder-specific options with an encoder.
After calling set_encoder_options, any calls to jsonpickle’s encode method will pass the supplied args and kwargs along to the appropriate backend’s encode method.
For example:
set_encoder_options('simplejson', sort_keys=True, indent=4) set_encoder_options('demjson', compactly=False)
See the appropriate encoder’s documentation for details about the supported arguments and keyword arguments.
-
set_preferred_backend
(name)¶ Set the preferred json backend.
If a preferred backend is set then jsonpickle tries to use it before any other backend.
For example:
set_preferred_backend('simplejson')
If the backend is not one of the built-in jsonpickle backends (json/simplejson, or demjson) then you must load the backend prior to calling set_preferred_backend.
AssertionError is raised if the backend has not been loaded.
jsonpickle.util
– Helper functions¶
Helper functions for pickling and unpickling. Most functions assist in determining the type of an object.
-
jsonpickle.util.
b64decode
(payload)¶ Decode payload - must be ascii text.
-
jsonpickle.util.
b64encode
(data)¶ Encode binary data to ascii text in base64. Data must be bytes.
-
jsonpickle.util.
b85decode
(payload)¶ Decode payload - must be ascii text.
-
jsonpickle.util.
b85encode
(data)¶ Encode binary data to ascii text in base85. Data must be bytes.
-
jsonpickle.util.
has_method
(obj, name)¶
-
jsonpickle.util.
has_reduce
(obj)¶ Tests if __reduce__ or __reduce_ex__ exists in the object dict or in the class dicts of every class in the MRO except object.
Returns a tuple of booleans (has_reduce, has_reduce_ex)
-
jsonpickle.util.
importable_name
(cls)¶ >>> class Example(object): ... pass
>>> ex = Example() >>> importable_name(ex.__class__) == 'jsonpickle.util.Example' True >>> importable_name(type(25)) == 'builtins.int' True >>> importable_name(None.__class__) == 'builtins.NoneType' True >>> importable_name(False.__class__) == 'builtins.bool' True >>> importable_name(AttributeError) == 'builtins.AttributeError' True
-
jsonpickle.util.
in_dict
(obj, key, default=False)¶ Returns true if key exists in obj.__dict__; false if not in. If obj.__dict__ is absent, return default
-
jsonpickle.util.
in_slots
(obj, key, default=False)¶ Returns true if key exists in obj.__slots__; false if not in. If obj.__slots__ is absent, return default
-
jsonpickle.util.
is_bytes
(obj)¶ Helper method to see if the object is a bytestring.
>>> is_bytes(b'foo') True
-
jsonpickle.util.
is_collections
(obj)¶
-
jsonpickle.util.
is_dictionary
(obj)¶ Helper method for testing if the object is a dictionary.
>>> is_dictionary({'key':'value'}) True
-
jsonpickle.util.
is_dictionary_subclass
(obj)¶ Returns True if obj is a subclass of the dict type. obj must be a subclass and not the actual builtin dict.
>>> class Temp(dict): pass >>> is_dictionary_subclass(Temp()) True
-
jsonpickle.util.
is_enum
(obj)¶ Is the object an enum?
-
jsonpickle.util.
is_function
(obj)¶ Returns true if passed a function
>>> is_function(lambda x: 1) True
>>> is_function(locals) True
>>> def method(): pass >>> is_function(method) True
>>> is_function(1) False
-
jsonpickle.util.
is_installed
(module)¶ Tests to see if
module
is available on the sys.path>>> is_installed('sys') True >>> is_installed('hopefullythisisnotarealmodule') False
-
jsonpickle.util.
is_iterator
(obj)¶
-
jsonpickle.util.
is_list
(obj)¶ Helper method to see if the object is a Python list.
>>> is_list([4]) True
-
jsonpickle.util.
is_list_like
(obj)¶
-
jsonpickle.util.
is_module
(obj)¶ Returns True if passed a module
>>> import os >>> is_module(os) True
-
jsonpickle.util.
is_module_function
(obj)¶ Return True if obj is a module-global function
>>> import os >>> is_module_function(os.path.exists) True
>>> is_module_function(lambda: None) False
-
jsonpickle.util.
is_noncomplex
(obj)¶ Returns True if obj is a special (weird) class, that is more complex than primitive data types, but is not a full object. Including:
-
jsonpickle.util.
is_object
(obj)¶ Returns True is obj is a reference to an object instance.
>>> is_object(1) True
>>> is_object(object()) True
>>> is_object(lambda x: 1) False
-
jsonpickle.util.
is_picklable
(name, value)¶ Return True if an object can be pickled
>>> import os >>> is_picklable('os', os) True
>>> def foo(): pass >>> is_picklable('foo', foo) True
>>> is_picklable('foo', lambda: None) False
-
jsonpickle.util.
is_primitive
(obj)¶ Helper method to see if the object is a basic data type. Unicode strings, integers, longs, floats, booleans, and None are considered primitive and will return True when passed into is_primitive()
>>> is_primitive(3) True >>> is_primitive([4,4]) False
-
jsonpickle.util.
is_reducible
(obj)¶ Returns false if of a type which have special casing, and should not have their __reduce__ methods used
-
jsonpickle.util.
is_sequence
(obj)¶ Helper method to see if the object is a sequence (list, set, or tuple).
>>> is_sequence([4]) True
-
jsonpickle.util.
is_sequence_subclass
(obj)¶ Returns True if obj is a subclass of list, set or tuple.
obj must be a subclass and not the actual builtin, such as list, set, tuple, etc..
>>> class Temp(list): pass >>> is_sequence_subclass(Temp()) True
-
jsonpickle.util.
is_set
(obj)¶ Helper method to see if the object is a Python set.
>>> is_set(set()) True
-
jsonpickle.util.
is_tuple
(obj)¶ Helper method to see if the object is a Python tuple.
>>> is_tuple((1,)) True
-
jsonpickle.util.
is_type
(obj)¶ Returns True is obj is a reference to a type.
>>> is_type(1) False
>>> is_type(object) True
>>> class Klass: pass >>> is_type(Klass) True
-
jsonpickle.util.
is_unicode
(obj)¶ Helper method to see if the object is a unicode string
-
jsonpickle.util.
itemgetter
(obj, getter=operator.itemgetter(0))¶
-
jsonpickle.util.
items
(obj)¶ Iterate over dicts in a deterministic order
Python2 does not guarantee dict ordering, so this function papers over the difference in behavior. Python3 does guarantee dict order, without use of OrderedDict, so no sorting is needed there.
-
jsonpickle.util.
translate_module_name
(module)¶ Rename builtin modules to a consistent module name.
Prefer the more modern naming.
This is used so that references to Python’s builtins module can be loaded in both Python 2 and 3. We remap to the “__builtin__” name and unmap it when importing.
Map the Python2 exceptions module to builtins because builtins is a superset and contains everything that is available in exceptions, which makes the translation simpler.
See untranslate_module_name() for the reverse operation.
-
jsonpickle.util.
untranslate_module_name
(module)¶ Rename module names mention in JSON to names that we can import
This reverses the translation applied by translate_module_name() to a module name available to the current version of Python.
Extensions¶
jsonpickle extensions¶
NumPy¶
jsonpickle includes a built-in numpy extension. If would like to encode sklearn models, numpy arrays, and other numpy-based data then you must enable the numpy extension by registering its handlers:
>>> import jsonpickle.ext.numpy as jsonpickle_numpy
>>> jsonpickle_numpy.register_handlers()
Contributing¶
Contributing to jsonpickle¶
We welcome contributions from everyone. Please fork jsonpickle on github.
Get the Code¶
git clone git://github.com/jsonpickle/jsonpickle.git
Run the Test Suite¶
Before code is pulled into the master jsonpickle branch, all tests should pass. If you are contributing an addition or a change in behavior, we ask that you document the change in the form of test cases.
The test suite is most readily run with the tox testing tool. Once installed, run the test suite against the default Python:
tox
It is recommended that you install at least one Python2 and one Python3 interpreter for use by tox. To test against Python 2.7 and 3.7:
tox -e py27,py37
The jsonpickle test suite uses several JSON encoding libraries as well as several libraries for sample objects. To create an environment to test against these libs:
tox -e libs
To test against these libs on Python 3.7:
tox -e py37-libs
To create the enivornment without running tests:
tox -e libs --notest
Now you may experiment and interact with jsonpickle under development
from the virtualenv at .tox/libs/{bin/Scripts}/python
.
Generate Documentation¶
Generating the documentation is not necessary when contributing. To build the docs:
tox -e docs
Now docs are available in build/html
.
If you wish to browse the documentation, use Python’s http.server
to host them at http://localhost:8000:
python -m http.server -d build/html
Contact¶
Please join our mailing list. You can send email to jsonpickle@googlegroups.com.
Check https://github.com/jsonpickle/jsonpickle for project updates.
Authors¶
John Paulett - john -at- paulett.org - https://github.com/johnpaulett
David Aguilar - davvid -at- gmail.com - https://github.com/davvid
Dan Buch - https://github.com/meatballhat
Ian Schenck - https://github.com/ianschenck
David K. Hess - https://github.com/davidkhess
Alec Thomas - https://github.com/alecthomas
jaraco - https://github.com/jaraco
Marcin Tustin - https://github.com/marcintustin
Change Log¶
History¶
v1.5.0¶
- 16 Jan 2021
Previous versions of jsonpickle with make_refs=False would emit
null
when encountering an object it had already seen when traversing objects. All instances of the object are now serialized. While this is arguably an improvement in the vast majority of scenarios, it is a change in behavior and is thus considered a minor-level change. (#333) (#334) (#337) (+338)Multiple enums are now serialized correctly with make_refs=False. (#235)
v1.4.2¶
v1.4¶
- 13 Apr 2020
Python 3.8 support. (#292)
jsonpickle.encode
now supports the standardindent
andseparators
arguments, and passes them through to the active JSON backend library. (#183)We now include a custom handler for array.array objects. (#199)
Dict key order is preserved when pickling dictionaries on Python3. (#193)
Improved serialization of dictionaries with non-string keys. Previously, using an enum that was both the key and a value in a dictionary could end up with incorrect references to other objects. The references are now properly maintained for dicts with object keys that are also referenced in the dict’s values. (#286)
Improved serialization of pandas.Series objects. (#287)
v1.3¶
v1.2¶
- 15 May 2019
Simplified JSON representation for __reduce__ values. (+261)
Improved Pandas support with new handlers for more Pandas data types. (+256)
Prevent stack overflows caused by bugs in user-defined __getstate__ functions which cause infinite recursion. (+260) (#259)
Improved support for objects that contain dicts with Integer keys. Previously, jsonpickle could not restore objects that contained dicts with integer keys and provided getstate only. These objects are now handled robustly. (#247).
Support for encoding binary data in base85 instead of base64 has been added on Python 3. Base85 produces payloads about 10% smaller than base64, albeit at the cost of lower throughput. For performance and backwards compatibility with Python 2 the pickler uses base64 by default, but it can be configured to use
base85
with the newuse_base85
argument. (#251).Dynamic SQLAlchemy tables in SQLAlchemy >= 1.3 are now supported. (#254).
v1.1¶
v1.0¶
- 01 Sep 2018
NOTE jsonpickle no longer supports Python2.6, or Python3 < 3.4. The officially supported Python versions are now 2.7 and 3.4+.
Improved Pandas and Numpy support. (+227)
Improved support for pickling iterators. (+216)
Better support for the stdlib json module when simplejson is not installed. (+217)
jsonpickle will now output python3-style module names when pickling builtins methods or functions. (+223)
jsonpickle will always flatten primitives, even when
max_depth
is reached, which avoids encoding unicode strings into theiru'string'
representation. (+207) (#180) (#198).Nested classes are now supported on Python 3. (+206) (#176).
Better support for older (pre-1.9) versions of numpy (+195).
v0.9.6¶
v0.9.5¶
- 16 Jul 2017
Better support for objects that implement the reduce protocol. (+170) This backward-incompatible change removes the SimpleReduceHandler. Any projects registering that handler for a particular type should instead remove references to the handler and jsonpickle will now handle those types directly.
v0.9.4¶
- 10 Jan 2017
Arbitrary byte streams are now better supported. (#143)
Better support for NumPy data types. The Python3 NumPy support is especially robust.
Fortran-ordered based NumPy arrays are now properly serialized.
v0.9.3¶
- 09 Mar 2016
UUID objects can now be serialized (#130)
Added set_decoder_options method to allow decoder specific options equal to set_encoder_options.
Int keys can be encoded directly by e.g. demjson by passing numeric_keys=True and setting its backend options via jsonpickle.set_encoder_options(‘demjson’, strict=False).
Newer Numpy versions (v1.10+) are now supported.
v0.9.2¶
- 20 Mar 2015
Fixes for serializing objects with custom handlers.
We now properly serialize deque objects constructed with a maxlen parameter.
Test suite fixes
v0.9.0¶
- 16 Jan 2015
Support for Pickle Protocol v4.
We now support serializing defaultdict subclasses that use self as their default factory.
We now have a decorator syntax for registering custom handlers, and allow custom handlers to register themselves for all subclasses. (+104)
We now support serializing types with metaclasses and their instances (e.g., Python 3 enum).
We now support serializing bytestrings in both Python 2 and Python 3. In Python 2, the str type is decoded to UTF-8 whenever possible and serialized as a true bytestring elsewise; in Python 3, bytestrings are explicitly encoded/decoded as bytestrings. Unicode strings are always encoded as is in both Python 2 and Python 3.
Added support for serializing numpy arrays, dtypes and scalars (see jsonpickle.ext.numpy module).
v0.8.0¶
06 Sep 2014
We now support serializing objects that contain references to module-level functions. (#77)
Better Pickle Protocol v2 support. (#78)
Support for string
__slots__
and iterable__slots__
. (#67) (#68)encode() now has a warn option that makes jsonpickle emit warnings when encountering objects that cannot be pickled.
A Javascript implementation of jsonpickle is now included in the jsonpickleJS directory.
v0.7.2¶
06 Aug 2014
We now properly serialize classes that inherit from classes that use __slots__ and add additional slots in the derived class.
jsonpickle can now serialize objects that implement __getstate__() but not __setstate__(). The result of __getstate__() is returned as-is when doing a round-trip from Python objects to jsonpickle and back.
Better support for collections.defaultdict with custom factories.
Added support for queue.Queue objects.
v0.7.0¶
15 Mar 2014
Added
handles
decorator tojsonpickle.handlers.BaseHandler
, enabling simple declaration of a handler for a class.__getstate__() and __setstate__() are now honored when pickling objects that subclass
dict
.jsonpickle can now serialize
collections.Counter
objects.Object references are properly handled when using integer keys.
Object references are now supported when using custom handlers.
Decimal objects are supported in Python 3.
jsonpickle’s “fallthrough-on-error” behavior can now be disabled.
Simpler API for registering custom handlers.
A new “safe-mode” is provided which avoids eval(). Backwards-compatible deserialization of repr-serialized objects is disabled in this mode. e.g. decode(string, safe=True)
v0.6.0¶
24 Aug 2013
Python 3 support!
time.struct_time
is now serialized using the built-injsonpickle.handlers.SimpleReduceHandler
.
v0.5.0¶
22 Aug 2013
Non-string dictionary keys (e.g. ints, objects) are now supported by passing keys=True to
jsonpickle.encode()
andjsonpickle.decode()
.We now support namedtuple, deque, and defaultdict.
Datetimes with timezones are now fully supported.
Better support for complicated structures e.g. datetime inside dicts.
jsonpickle added support for references and cyclical data structures in 0.4.0. This can be disabled by passing make_refs=False to
jsonpickle.encode()
.
0.4.0¶
21 Jun 2011
Switch build from setuptools to distutils
Consistent dictionary key ordering
Fix areas with improper support for unpicklable=False
Added support for cyclical data structures (#16).
New contributers David K. Hess and Alec Thomas
Warning
To support cyclical data structures (#16), the storage format has been modified. Efforts have been made to ensure backwards-compatibility. jsonpickle 0.4.0 can read data encoded by jsonpickle 0.3.1, but earlier versions of jsonpickle may be unable to read data encoded by jsonpickle 0.4.0.
0.3.0¶
11 Dec 2009
Officially migrated to git from subversion. Project home now at http://jsonpickle.github.com/. Thanks to Michael Jone’s sphinx-to-github.
Fortified jsonpickle against common error conditions.
Added support for:
List and set subclasses.
Objects with module references.
Newstyle classes with __slots__.
Objects implementing __setstate__() and __getstate__() (follows the
pickle
protocol).
Improved support for Zope objects via pre-fetch.
Support for user-defined serialization handlers via the jsonpickle.handlers registry.
Removed cjson support per John Millikin’s recommendation.
General improvements to style, including PEP 257 compliance and refactored project layout.
Steps towards Python 2.3 and Python 3 support.
New contributors Dan Buch and Ian Schenck.
Thanks also to Kieran Darcy, Eoghan Murray, and Antonin Hildebrand for their assistance!
0.2.0¶
10 Jan 2009
Support for all major Python JSON backends (including json in Python 2.6, simplejson, cjson, and demjson)
Handle several datetime objects using the repr() of the objects (Thanks to Antonin Hildebrand).
Sphinx documentation
Added support for recursive data structures
Unicode dict-keys support
Support for Google App Engine and Django
Tons of additional testing and bug reports (Antonin Hildebrand, Sorin, Roberto Saccon, Faber Fedor, FirePython, and Joose)
0.1.0¶
22 Aug 2008
Added long as basic primitive (thanks Adam Fisk)
Prefer python-cjson to simplejson, if available
Major API change, use python-cjson’s decode/encode instead of simplejson’s load/loads/dump/dumps
Added benchmark.py to compare simplejson and python-cjson
0.0.5¶
22 Jul 2008
Changed prefix of special fields to conform with CouchDB requirements (Thanks Dean Landolt). Break backwards compatibility.
Moved to Google Code subversion
Fixed unit test imports
0.0.3¶
Convert back to setup.py from pavement.py (issue found by spidaman)
0.0.2¶
Handle feedparser’s FeedParserDict
Converted project to Paver
Restructured directories
Increase test coverage
0.0.1¶
Initial release
License¶
jsonpickle is provided under a New BSD license,
Copyright (C) 2008-2011 John Paulett (john -at- paulett.org) Copyright (C) 2009-2016 David Aguilar (davvid -at- gmail.com)