jds

Jenesis Data Store: a dynamic, cross platform, high performance, ORM data-mapper. Designed to assist in rapid development and data mining

Programming language: Kotlin

License: GNU General Public License v3.0 or later

Tags: Database    

Latest version: v20.4

README

[](LICENSE.md) <!--- )--->

Jenesis Data Store

Jenesis Data Store (JDS) was created to help developers persist data to a strongly-typed portable JSON format.

JDS has four goals:

• To allow for the rapid development of complex Java/Kotlin systems with stringent data definition / quality requirements
• To provide a flexible and reliable framework to persist and retrieve data against.
• To provide a robust, strongly-typed Field Dictionary.
• To leverage JSON as a datastore over EAV based paradigms.

The library eliminates the need to modify schemas once a class has been altered.

It also eliminates all concerns regarding "breaking changes" in regards to fields and their addition and/or removal.

Put simply, JDS is useful for any developer that requires a flexible data store running on top of a Relational databases.

JDS is licensed under the 3-Clause BSD License

Design

The concept behind JDS is quite simple. Extend a base Entity class, define strongly-typed Fields and then map them against implementations of the Property interface.

JDS was designed to avoid reflection and its potential performance pitfalls. As such mapping functions which are overridden, and invoked at least once at runtime, are used to enforce/validate typing as instead of annotations. This is discussed below in section 1.1.4 "Binding properties".

Features

• Transparent persistence
• Supports the persistence of NULL values for boxed types (e.g Integer, Double)
• Full support for generics and inheritance
• Easily integrates with new or existing databases
• Portable format which can be serialised to JSON allows for the flexibility of EAV without its drawbacks
• Supports MySQL, T-SQL, PostgreSQL, Oracle 11G, MariaDB and SQLite
• Supports a robust Field Dictonary allowing for metadata such as Tags and Alternate Coding to be applied to Fields and Entities.

Maven Central

You can search on The Central Repository with GroupId and ArtifactId Maven Search for

Maven

<dependency>
    <groupId>io.github.subiyacryolite</groupId>
    <artifactId>jds</artifactId>
    <version>20.4</version>
</dependency>

Gradle

compile 'io.github.subiyacryolite:jds:20.4'

Dependencies

The library depends on Java 1.8. Both 64 and 32 bit variants should suffice. Both the Development Kit and Runtime can be downloaded from here.

Supported Databases

The API currently supports the following Relational Databases, each of which has their own dependencies, versions and licensing requirements. Please consult the official sites for specifics.

Database	Version Tested Against	Official Site	JDBC Driver Tested Against
PostgreSQL	9.5	Official Site	org.postgresql
MySQL	5.7.14	Official Site	com.mysql.cj.jdbc.Driver
MariaDb	10.2.12	Official Site	org.mariadb.jdbc.Driver
Microsoft SQL Server	2008 R2	Official Site	com.microsoft.sqlserver
SQLite	3.16.1	Official Site	org.sqlite.JDBC
Oracle	11g Release 2	Official Site	oracle.jdbc.driver.OracleDriver

1 How it works

1.1 Creating Classes

Classes that use JDS need to extend Entity.

import io.github.subiyacryolite.jds.Entity;

public class Address : Entity

However, if you plan on using interfaces they must extend IEntity. Concrete classes can then extend Entity

import io.github.subiyacryolite.jds.Entity;
import io.github.subiyacryolite.jds.IEntity;

public interface IAddress : IEntity

public class Address : IAddress

1.1.1 Annotating Classes

Every class/interface which extends Entity/IEntity must have its own unique Entity ID as well as an Entity Name. This is done by annotating the class, or its (parent) interface.

@EntityAnnotation(id = 1, name = "address", description = "An entity representing address information")
class Address : Entity()

Entity IDs MUST be unique in your application, any value of type long is valid. Entity Names do not enforce unique constraints but its best to use a unique name regardless. These values can be referenced to mine data.

1.1.2 Defining Fields

Fields are big part of the JDS framework. Each Field MUST have a unique Field Id. Field Names do not enforce unique constraints but its best to use a unique name regardless. These values can be referenced to mine data. Every Field that you define can be one of the following types.

JDS Field Type	Java Type	Description
DateTimeCollection	Collection<LocalDateTime>	Collection of type LocalDateTime
DoubleCollection	Collection<Double>	Collection of type Double
EntityCollection	Collection<Class<? extends Entity>>	Collection of type Entity
FloatCollection	Collection<Float>	Collection of type Float
IntCollection	Collection<Integer>	Collection of type Integer
ShortCollection	Collection<Short>	Collection of type Short
LongCollection	Collection<Long>	Collection of type Long
StringCollection	Collection<String>	Collection of type String
UuidCollection	Collection<UUID>	Collection of type UUID
Blob	byte[] or InputStream	Blob values
Boolean	boolean / Boolean	Boolean values
Entity	Class<? extends Entity>	Object of type Entity
DateTime	LocalDateTime	DateTime instances based on the host machines local timezone
Date	LocalDate	Local date instances
Double	double / Double	Numeric double values
Duration	Duration	Object of type Duration
EnumCollection	Collection<Enum>	Collection of type Enum
Enum	Enum	Object of type Enum
Float	float / Float	Numeric float values
Int	int / Integer	Numeric integer values
Short	short / Short	Numeric SHORT values
Long	long / Long	Numeric long values
MonthDay	MonthDay	Object of type MonthDay
Period	Period	Object of type Period
String	String	String values with no max limit
Time	LocalTime	Local time instances
YearMonth	YearMonth	Object of type YearMonth
ZonedDateTime	ZonedDateTime	Zoned DateTime instances
Uuid	UUID	UUID instances

We recommend defining your Fields as static constants

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String)
    val PlotNumber = Field(2, "plot_number", FieldType.Int)
    val Area = Field(3, "area_name", FieldType.String)
    val ProvinceOrState = Field(4, "province_name", FieldType.String)
    val City = Field(5, "city_name", FieldType.String)
    val Country = Field(7, "country_name", FieldType.String)
    val PrimaryAddress = Field(8, "primary_address", FieldType.Boolean)
    val Timestamp = Field(9, "timestamp", FieldType.DateTime)
}

Furthermore, you can add descriptions, of up to 256 characters, to each field

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String, "The street name of the address")
    val PlotNumber = Field(2, "plot_number", FieldType.Int, "The street name of the address")
    val Area = Field(3, "area_name", FieldType.String, "The name of the area / neighbourhood")
    //...
}

JDS also supports Tags which can be applied to each Field and Entity definitions. Tags can be defined as a set of strings, there is no limit on how many tags a field can have. This can be useful for categorising certain kinds of information

import io.github.subiyacryolite.jds.Field;
import io.github.subiyacryolite.jds.enumProperties.FieldType;

object Fields {
    val StreetName = Field(1, "street_name", FieldType.String, description = "The street name of the address", tags = setOf("AddressInfo", "ClientInfo", "IdentifiableInfo"))
    //...
}

1.1.3 Defining Enums

JDS Enums are an extension of Fields. Usually these values would be represented by Check Boxes, Radio Buttons or Combo Boxes on the front-end.

First we'd define a standard JDS Field of type Enum.

import io.github.subiyacryolite.jds.Field
import io.github.subiyacryolite.jds.enums.FieldType

public class Fields
{
    val Direction = Field(10, "direction", FieldType.Enum)
}

Then, we can define our actual enum in the following manner.

enum class Direction {
    North, West, South, East
}

Lastly, we create an instance of the JDS Field Enum type.

import io.github.subiyacryolite.jds.FieldEnum

object Enums {
    val Directions = FieldEnum(Direction::class.java, Fields.Direction, *Direction.values())
}

Behind the scenes these Enums will be stored as either:

• an Integer (FieldType.Enum);
• a String (FieldType.EnumString);
• an Integer Array (FieldType.EnumCollection); or
• a String Collection (FieldType.EnumStringCollection)

1.1.4 Binding Properties

Depending on the type of Field, JDS will require that you set you objects properties to one of the following IValue container types.

JDS Field Type	Container	Java Mapping Call	Kotlin Mapping Call
DateTimeCollection	MutableCollection<LocalDateTime>	mapDateTimes	map
DoubleCollection	MutableCollection<Double>	mapDoubles	map
EntityCollection	MutableCollection<Class<? extends Entity>>	map	map
FloatCollection	MutableCollection<Float>	mapFloats	map
IntCollection	MutableCollection<Integer>	mapInts	map
LongCollection	MutableCollection<Long>	mapLongs	map
StringCollection	MutableCollection<String>	mapStrings	map
Boolean	IValue<Boolean>	mapBoolean	map
Blob	IValue<ByteArray>	map	map
Entity	Class<? extends Entity>	map	map
Date	IValue<LocalDate>	mapDate	map
DateTime	IValue<LocalDateTime>	mapDateTime	map
Double	IValue<Double>	mapNumeric	map
Duration	IValue<Duration>	mapDuration	map
Enum	IValue<Enum>	mapEnum	map
EnumCollection	Collection<Enum>	mapEnums	map
Float	IValue<Float>	mapNumeric	map
Int	IValue<Integer>	mapNumeric	map
Long	IValue<Long>	mapNumeric	map
MonthDay	IValue<MonthDay>	mapMonthDay	map
Period	IValue<Period>	mapPeriod	map
String	IValue<String>	mapString	map
Time	IValue<LocalTime>	mapTime	map
YearMonth	IValue<YearMonth>	mapYearMonth	map
ZonedDateTime	IValue<ZonedDateTime>	mapZonedDateTime	map
Uuid	IValue<ZonedDateTime>	mapZonedDateTime	map

To simplify the mapping Process Jds has the following helper classes defined:

• Generic containers (Entities and Enums)
  • ObjectValue
• Non null containers
  • BlobValue
  • BooleanValue
  • DoubleValue
  • DurationValue
  • EnumValue
  • FloatValue
  • IntegerValue
  • LocalDateValue
  • LocalDateTimeValue
  • LocalTimeValue
  • LongValue
  • MonthDayValue
  • PeriodValue
  • ShortValue
  • StringValue
  • UuidValue
  • YearMonthValue
  • ZonedDateTimeValue
• Nullable containers
  • NullableBlobValue
  • NullableBooleanValue
  • NullableDoubleValue
  • NullableDurationValue
  • NullableEnumValue
  • NullableFloatValue
  • NullableIntegerValue
  • NullableLocalDateValue
  • NullableLocalDateTimeValue
  • NullableLocalTimeValue
  • NullableLongValue
  • NullableMonthDayValue
  • NullablePeriodValue
  • NullableShortValue
  • NullableStringValue
  • NullableUuidValue
  • NullableYearMonthValue
  • NullableZonedDateTimeValue

Note: JDS assumes that all collection types will not contain null entries.

Note: Collection types can be of any valid type e.g. ArrayList, LinkedList, HashSet etc

After your class and its properties have been defined you must map the property to its corresponding Field using the map() method. I recommend doing this in your primary constructor.

The example below shows a class definition with valid properties and bindings. With this your class can be persisted.

Note that the example below has a 3rd parameter to the map method, this is the Property Name

The Property Name is used by the JDS Field Dictionary to know which property a particular Field is mapped to within an Entity.

This is necessary as one Field definition can be mapped to a different property amongst different Entities.

For example a Field called "FirstName" could be mapped to a property called "firstName" in one Entity and a property called "givenName" in another.

package io.github.subiyacryolite.jds.tests.entities

import io.github.subiyacryolite.jds.Entity
import io.github.subiyacryolite.jds.annotations.EntityAnnotation
import io.github.subiyacryolite.jds.beans.property.NullableBooleanValue
import io.github.subiyacryolite.jds.beans.property.NullableShortValue
import io.github.subiyacryolite.jds.tests.constants.Fields
import java.time.LocalDateTime

data class Address(
        private val _streetName: IValue<String> = StringValue(),
        private val _plotNumber: IValue<Short?> = NullableShortValue(),
        private val _area: IValue<String> = StringValue(),
        private val _city: IValue<String> = StringValue(),
        private val _provinceOrState: IValue<String> = StringValue(),
        private val _country: IValue<String> = StringValue(),
        private val _primaryAddress: IValue<Boolean?> = NullableBooleanValue(),
        private val _timestamp: IValue<LocalDateTime> = LocalDateTimeValue()
) : Entity(), IAddress {

    override fun bind() {
        super.bind()
        map(Fields.StreetName, _streetName, "streetName")
        map(Fields.PlotNumber, _plotNumber, "plotNumber")
        map(Fields.ResidentialArea, _area, "area")
        map(Fields.City, _city, "city")
        map(Fields.ProvinceOrState, _provinceOrState, "provinceOrState")
        map(Fields.Country, _country, "country")
        map(Fields.PrimaryAddress, _primaryAddress, "primaryAddress")
        map(Fields.TimeStamp, _timestamp, "timestamp")
    }

    var primaryAddress: Boolean?
        get() = _primaryAddress.get()
        set(value) = _primaryAddress.set(value)

    var streetName: String
        get() = _streetName.get()
        set(value) = _streetName.set(value)

    var plotNumber: Short?
        get() = _plotNumber.get()
        set(value) = _plotNumber.set(value)

    var area: String
        get() = _area.get()
        set(value) = _area.set(value)

    var city: String
        get() = _city.get()
        set(value) = _city.set(value)

    var provinceOrState: String
        get() = _provinceOrState.get()
        set(value) = _provinceOrState.set(value)

    var country: String
        get() = _country.get()
        set(value) = _country.set(value)

    var timeOfEntry: LocalDateTime
        get() = _timestamp.get()
        set(timeOfEntry) = _timestamp.set(timeOfEntry)
}

1.1.5 Binding Objects and Object Arrays

JDS can also persist embedded objects and object arrays.

All that's required is a valid Entity or IEntity subclass to be mapped to a Field of type Entity or EntityCollection .

import io.github.subiyacryolite.jds.Field
import io.github.subiyacryolite.jds.enums.FieldType

object Fields
{
    val Addresses = Field(23, "addresses", FieldType.EntityCollection, "A collection of addresses")
}

import io.github.subiyacryolite.jds.FieldEntity

object Entities {
    val Addresses: FieldEntity<Address> = FieldEntity(Address::class.java, Fields.Addresses)
}

import io.github.subiyacryolite.jds.Entity
import io.github.subiyacryolite.jds.annotations.EntityAnnotation
import io.github.subiyacryolite.jds.tests.constants.Entities

@EntityAnnotation(id = 2, name = "address_book")
data class AddressBook(
        val addresses: MutableCollection<IAddress> = ArrayList()
) : Entity() {

    override fun bind() {
        super.bind()
        map(Entities.Addresses, addresses, "addresses")
    }
}

1.2 CRUD Operations

1.2.1 Initialising the database

In order to use JDS you will need an instance of DbContext. Your instance of DbContext will have to extend one of the following classes:

• MariaDbContext;
• MySqlContext;
• OracleContext;
• SqLiteDbContext; or
• TransactionalSqlContext

After this you must override the dataSource property.

Please note that your project must have the correct JDBC driver in its class path. The drivers that were used during development are listed under Supported Databases above.

These samples use HikariCP to provide connection pooling for enhanced performance.

Postgres example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.PostGreSqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class PostGreSqlContextImplementation : PostGreSqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.pg.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:postgresql://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

MySQL Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.MySqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class MySqlContextImplementation : MySqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.mysql.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:mysql://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

MariaDb Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.MariaDbContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class MariaDbContextImplementation : MariaDbContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.mariadb.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:mariadb://${properties["dbUrl"]}:${properties["dbPort"]}/${properties["dbName"]}"

        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Microsoft SQL Server Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.TransactionalSqlContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class TransactionalSqlContextImplementation : TransactionalSqlContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.tsql.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.jdbcUrl = "jdbc:sqlserver://${properties["dbUrl"]}\\${properties["dbInstance"]};databaseName=${properties["dbName"]}"
        hikariConfig.dataSourceProperties = properties //additional props
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Oracle Example

import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import io.github.subiyacryolite.jds.context.OracleContext
import java.io.File
import java.io.FileInputStream
import java.util.*
import javax.sql.DataSource

class OracleContextImplementation : OracleContext() {

    private val properties: Properties = Properties()
    private val hikariDataSource: DataSource

    init {
        FileInputStream(File("db.ora.properties")).use { properties.load(it) }

        val hikariConfig = HikariConfig()
        hikariConfig.driverClassName = properties["driverClassName"].toString()
        hikariConfig.maximumPoolSize = properties["maximumPoolSize"].toString().toInt()
        hikariConfig.username = properties["username"].toString()
        hikariConfig.password = properties["password"].toString()
        hikariConfig.dataSourceProperties = properties //additional props
        hikariConfig.jdbcUrl = "jdbc:oracle:thin:@${properties["dbUrl"]}:${properties["dbPort"]}:${properties["dbName"]}"
        hikariDataSource = HikariDataSource(hikariConfig)
    }

    override val dataSource: DataSource
        get () = hikariDataSource
}

Sqlite Example

import io.github.subiyacryolite.jds.context.SqLiteDbContext
import org.sqlite.SQLiteConfig
import org.sqlite.SQLiteDataSource
import java.io.File
import javax.sql.DataSource

class SqLiteDbContextImplementation : SqLiteDbContext() {

    private val sqLiteDataSource: DataSource

    init {
        val path = File(System.getProperty("user.home") + File.separator + ".jdstest" + File.separator + "jds.db")
        if (!path.exists())
            if (!path.parentFile.exists())
                path.parentFile.mkdirs()
        val sqLiteConfig = SQLiteConfig()
        sqLiteConfig.enforceForeignKeys(true) //You must enable foreign keys in SQLite
        sqLiteDataSource = SQLiteDataSource(sqLiteConfig)
        sqLiteDataSource.url = "jdbc:sqlite:${path.absolutePath}"
    }

    override val dataSource: DataSource
        get () {
            Class.forName("org.sqlite.JDBC")
            return sqLiteDataSource
        }
}

With this you should have a valid data source allowing you to access your database. JDS will automatically setup its tables and procedures at runtime.

Furthermore, you can use the getConnection() method OR connection property from this dataSource property in order to return a standard java.sql.Connection in your application.

1.2.2 Initialising JDS

Once you have initialised your database you can go ahead and initialise all your JDS classes. You can achieve this by mapping ALL your JDS classes in the following manner.

fun initialiseJdsClasses(dbContext: DbContext)
{
    dbContext.map(Address::class.java);
    dbContext.map(AddressBook::class.java);
}

You only have to do this once at start-up. Without this you will not be able to persist or load data.

1.2.3 Creating objects

Once you have defined your class you can initialise them. A dynamic id is created for every Entity by default (using javas UUID class). This value is used to uniquely identify an object and it data in the database. You can set your own values if you wish.

    val primaryAddress = Address()
    primaryAddress.overview.id = "primaryAddress" //explicit id defined, JDS assigns a value by default on instantiation
    primaryAddress.area = "Norte Broad"
    primaryAddress.city = "Livingstone"
    primaryAddress.country = "Zambia"
    primaryAddress.plotNumber = null
    primaryAddress.provinceOrState = "Southern"
    primaryAddress.streetName = "East Street"
    primaryAddress.timeOfEntry = LocalTime.now()
    primaryAddress.primaryAddress = PrimaryAddress.YES

1.2.4 Saving objects (Portable Format)

...

1.2.5 Loading objects (Portable Format)

...

Development

I highly recommend the use of the IntelliJ IDE for development.

Contributing to Jenesis Data Store

If you would like to contribute code you can do so through Github by forking the repository and sending a pull request targeting the current development branch.

When submitting code, please make every effort to follow existing conventions and style in order to keep the code as readable as possible.

Bugs and Feedback

For bugs, questions and discussions please use the Github Issues.

Special Thanks

To all our users and contributors!