@@ -1208,87 +1208,90 @@ Limit on the complexity of SQL statements. Statements of the following form will
...
@@ -1208,87 +1208,90 @@ Limit on the complexity of SQL statements. Statements of the following form will
There is no limit for the following entities, except the memory and storage capacity: maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
There is no limit for the following entities, except the memory and storage capacity: maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
@advanced_1403_li
@advanced_1403_li
Querying from the metadata tables is slow if there are many tables (thousands).
@advanced_1404_li
For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
@advanced_1404_h2
@advanced_1405_h2
Glossary and Links
Glossary and Links
@advanced_1405_th
@advanced_1406_th
Term
Term
@advanced_1406_th
@advanced_1407_th
Description
Description
@advanced_1407_td
@advanced_1408_td
AES-128
AES-128
@advanced_1408_td
@advanced_1409_td
A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia: AES</a>
A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia: AES</a>
@advanced_1409_td
@advanced_1410_td
Birthday Paradox
Birthday Paradox
@advanced_1410_td
@advanced_1411_td
Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also: <a href="http://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia: Birthday Paradox</a>
Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also: <a href="http://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia: Birthday Paradox</a>
@advanced_1411_td
@advanced_1412_td
Digest
Digest
@advanced_1412_td
@advanced_1413_td
Protocol to protect a password (but not to protect data). See also: <a href="http://www.faqs.org/rfcs/rfc2617.html">RFC 2617: HTTP Digest Access Authentication</a>
Protocol to protect a password (but not to protect data). See also: <a href="http://www.faqs.org/rfcs/rfc2617.html">RFC 2617: HTTP Digest Access Authentication</a>
@advanced_1413_td
@advanced_1414_td
GCJ
GCJ
@advanced_1414_td
@advanced_1415_td
Compiler for Java. <a href="http://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href="http://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
Compiler for Java. <a href="http://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href="http://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
@advanced_1415_td
@advanced_1416_td
HTTPS
HTTPS
@advanced_1416_td
@advanced_1417_td
A protocol to provide security to HTTP connections. See also: <a href="http://www.ietf.org/rfc/rfc2818.txt">RFC 2818: HTTP Over TLS</a>
A protocol to provide security to HTTP connections. See also: <a href="http://www.ietf.org/rfc/rfc2818.txt">RFC 2818: HTTP Over TLS</a>
@advanced_1417_td
@advanced_1418_td
Modes of Operation
Modes of Operation
@advanced_1418_a
@advanced_1419_a
Wikipedia: Block cipher modes of operation
Wikipedia: Block cipher modes of operation
@advanced_1419_td
@advanced_1420_td
Salt
Salt
@advanced_1420_td
@advanced_1421_td
Random number to increase the security of passwords. See also: <a href="http://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia: Key derivation function</a>
Random number to increase the security of passwords. See also: <a href="http://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia: Key derivation function</a>
@advanced_1421_td
@advanced_1422_td
SHA-256
SHA-256
@advanced_1422_td
@advanced_1423_td
A cryptographic one-way hash function. See also: <a href="http://en.wikipedia.org/wiki/SHA_family">Wikipedia: SHA hash functions</a>
A cryptographic one-way hash function. See also: <a href="http://en.wikipedia.org/wiki/SHA_family">Wikipedia: SHA hash functions</a>
@advanced_1423_td
@advanced_1424_td
SQL Injection
SQL Injection
@advanced_1424_td
@advanced_1425_td
A security vulnerability where an application embeds SQL statements or expressions in user input. See also: <a href="http://en.wikipedia.org/wiki/SQL_injection">Wikipedia: SQL Injection</a>
A security vulnerability where an application embeds SQL statements or expressions in user input. See also: <a href="http://en.wikipedia.org/wiki/SQL_injection">Wikipedia: SQL Injection</a>
@advanced_1425_td
@advanced_1426_td
Watermark Attack
Watermark Attack
@advanced_1426_td
@advanced_1427_td
Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
@advanced_1427_td
@advanced_1428_td
SSL/TLS
SSL/TLS
@advanced_1428_td
@advanced_1429_td
Secure Sockets Layer / Transport Layer Security. See also: <a href="http://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
Secure Sockets Layer / Transport Layer Security. See also: <a href="http://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
@advanced_1429_td
@advanced_1430_td
XTEA
XTEA
@advanced_1430_td
@advanced_1431_td
A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/XTEA">Wikipedia: XTEA</a>
A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/XTEA">Wikipedia: XTEA</a>
@build_1000_h1
@build_1000_h1
...
@@ -3848,7 +3851,7 @@ Ignore Unknown Settings
...
@@ -3848,7 +3851,7 @@ Ignore Unknown Settings
Changing Other Settings when Opening a Connection
Changing Other Settings when Opening a Connection
@features_1389_p
@features_1389_p
In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href="grammar.html">SQL Grammar</a>.
In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href="grammar.html">SQL Grammar</a> or the <a href="../javadoc/org/h2/constant/DbSettings.html">DbSettings</a> javadoc.
@features_1390_h2
@features_1390_h2
Custom File Access Mode
Custom File Access Mode
...
@@ -6799,301 +6802,346 @@ MVStore
...
@@ -6799,301 +6802,346 @@ MVStore
@mvstore_1005_a
@mvstore_1005_a
Features
Features
@mvstore_1006_a
@mvstore_1006_div
- <a href="#maps">Maps</a>
@mvstore_1007_div
- <a href="#versions">Versions</a>
@mvstore_1008_div
- <a href="#transactions">Transactions</a>
@mvstore_1009_div
- <a href="#inMemory">In-Memory Performance and Usage</a>
@mvstore_1010_div
- <a href="#dataTypes">Pluggable Data Types</a>
@mvstore_1011_div
- <a href="#blob">BLOB Support</a>
@mvstore_1012_div
- <a href="#pluggableMap">R-Tree and Pluggable Map Implementations</a>
@mvstore_1013_div
- <a href="#caching">Concurrent Operations and Caching</a>
Similar Projects and Differences to Other Storage Engines
Similar Projects and Differences to Other Storage Engines
@mvstore_1007_a
@mvstore_1020_a
Current State
Current State
@mvstore_1008_a
@mvstore_1021_a
Requirements
Requirements
@mvstore_1009_h2
@mvstore_1022_h2
Overview
Overview
@mvstore_1010_p
@mvstore_1023_p
The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
@mvstore_1011_li
@mvstore_1024_li
MVStore stands for "multi-version store".
MVStore stands for "multi-version store".
@mvstore_1012_li
@mvstore_1025_li
Each store contains a number of maps (using the <code>java.util.Map</code> interface).
Each store contains a number of maps (using the <code>java.util.Map</code> interface).
@mvstore_1013_li
@mvstore_1026_li
Both file-based persistence and in-memory operation are supported.
Both file-based persistence and in-memory operation are supported.
@mvstore_1014_li
@mvstore_1027_li
It is intended to be fast, simple to use, and small.
It is intended to be fast, simple to use, and small.
@mvstore_1015_li
@mvstore_1028_li
Old versions of the data can be read concurrently with all other operations.
Old versions of the data can be read concurrently with all other operations.
@mvstore_1016_li
@mvstore_1029_li
Transaction are supported (currently only one transaction at a time).
Transaction are supported.
@mvstore_1017_li
Transactions (even if they are persisted) can be rolled back.
@mvstore_1018_li
@mvstore_1030_li
The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
@mvstore_1019_h2
@mvstore_1031_h2
Example Code
Example Code
@mvstore_1020_p
@mvstore_1032_p
The following sample code show how to create a store, open a map, add some data, and access the current and an old version:
The following sample code show how to create a store, open a map, add some data, and access the current and an old version:
@mvstore_1021_h2
@mvstore_1033_h2
Store Builder
Store Builder
@mvstore_1022_p
@mvstore_1034_p
The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options:
The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options:
@mvstore_1023_li
@mvstore_1035_li
cacheSizeMB: the cache size in MB.
cacheSizeMB: the cache size in MB.
@mvstore_1024_li
@mvstore_1036_li
compressData: compress the data when storing.
compressData: compress the data when storing.
@mvstore_1025_li
@mvstore_1037_li
encryptionKey: the encryption key for file encryption.
encryptionKey: the encryption key for file encryption.
@mvstore_1026_li
@mvstore_1038_li
fileName: the name of the file, for file based stores.
fileName: the name of the file, for file based stores.
@mvstore_1027_li
@mvstore_1039_li
readOnly: open the file in read-only mode.
readOnly: open the file in read-only mode.
@mvstore_1028_li
@mvstore_1040_li
writeBufferSize: the size of the write buffer in MB.
writeBufferSize: the size of the write buffer in MB.
@mvstore_1029_li
@mvstore_1041_li
writeDelay: the maximum delay until committed changes are stored (unless stored explicitly).
writeDelay: the maximum delay until committed changes are stored (unless stored explicitly).
@mvstore_1030_h2
@mvstore_1042_h2
R-Tree
R-Tree
@mvstore_1031_p
@mvstore_1043_p
The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows:
The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows:
@mvstore_1032_p
@mvstore_1044_p
The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
@mvstore_1033_h2
@mvstore_1045_h2
Features
Features
@mvstore_1034_h3
@mvstore_1046_h3
Maps
Maps
@mvstore_1035_p
@mvstore_1047_p
Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
@mvstore_1036_p
@mvstore_1048_p
Also supported, and very uncommon for maps, is fast index lookup: the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
Also supported, and very uncommon for maps, is fast index lookup: the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
@mvstore_1037_p
@mvstore_1049_p
In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
@mvstore_1038_h3
@mvstore_1050_h3
Versions / Transactions
Versions
@mvstore_1039_p
@mvstore_1051_p
Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
@mvstore_1040_p
@mvstore_1052_p
Versions / transactions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
Versions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
@mvstore_1041_p
@mvstore_1053_p
Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast: only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast: only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
@mvstore_1042_p
@mvstore_1054_p
Rollback is supported (rollback to any old in-memory version or an old persisted version).
Rollback is supported (rollback to any old in-memory version or an old persisted version).
@mvstore_1043_h3
@mvstore_1055_h3
Transactions
@mvstore_1056_p
The multi-version support is the basis for the transaction support. In the simple case, when only one transaction is open at a time, rolling back the transaction only requires to revert to an old version.
@mvstore_1057_p
To support multiple concurrent open transactions, a transaction utility is included, the <code>TransactionStore</code>. This utility stores the changed entries in a separate map, similar to a transaction log (except that only the key of a changed row is stored, and the entries of a transaction are removed when the transaction is committed). The storage overhead of this utility is very small compared to the overhead of a regular transaction log. The tool supports PostgreSQL style "read committed" transaction isolation. There is no limit on the size of a transaction (the log is not kept in memory).
@mvstore_1058_h3
In-Memory Performance and Usage
In-Memory Performance and Usage
@mvstore_1044_p
@mvstore_1059_p
Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
@mvstore_1045_p
@mvstore_1060_p
The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
@mvstore_1046_p
@mvstore_1061_p
If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
@mvstore_1047_h3
@mvstore_1062_h3
Pluggable Data Types
Pluggable Data Types
@mvstore_1048_p
@mvstore_1063_p
Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported: <code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported: <code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
@mvstore_1049_p
@mvstore_1064_p
Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
@mvstore_1050_p
@mvstore_1065_p
The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
@mvstore_1051_h3
@mvstore_1066_h3
BLOB Support
BLOB Support
@mvstore_1052_p
@mvstore_1067_p
There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
@mvstore_1053_h3
@mvstore_1068_h3
R-Tree and Pluggable Map Implementations
R-Tree and Pluggable Map Implementations
@mvstore_1054_p
@mvstore_1069_p
The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
@mvstore_1055_h3
@mvstore_1070_h3
Concurrent Operations and Caching
Concurrent Operations and Caching
@mvstore_1056_p
@mvstore_1071_p
The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
@mvstore_1057_p
@mvstore_1072_p
Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
@mvstore_1058_p
@mvstore_1073_p
Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
@mvstore_1059_p
@mvstore_1074_p
The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
@mvstore_1060_p
@mvstore_1075_p
With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
@mvstore_1061_p
@mvstore_1076_p
For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
@mvstore_1062_h3
@mvstore_1077_h3
Log Structured Storage
Log Structured Storage
@mvstore_1063_p
@mvstore_1078_p
Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
@mvstore_1064_p
@mvstore_1079_p
When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index: all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index: all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
@mvstore_1065_p
@mvstore_1080_p
There are usually two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
There are usually two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
@mvstore_1066_p
@mvstore_1081_p
There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
@mvstore_1067_p
@mvstore_1082_p
Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
@mvstore_1068_p
@mvstore_1083_p
Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
@mvstore_1069_h3
@mvstore_1084_h3
File System Abstraction, File Locking and Online Backup
File System Abstraction, File Locking and Online Backup
@mvstore_1070_p
@mvstore_1085_p
The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
@mvstore_1071_p
@mvstore_1086_p
Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
@mvstore_1072_p
@mvstore_1087_p
The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
@mvstore_1073_h3
@mvstore_1088_h3
Encrypted Files
Encrypted Files
@mvstore_1074_p
@mvstore_1089_p
File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows:
File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows:
@mvstore_1075_p
@mvstore_1090_p
The following algorithms and settings are used:
The following algorithms and settings are used:
@mvstore_1076_li
@mvstore_1091_li
The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
@mvstore_1077_li
@mvstore_1092_li
The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
@mvstore_1078_li
@mvstore_1093_li
The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
@mvstore_1079_li
@mvstore_1094_li
To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
@mvstore_1080_li
@mvstore_1095_li
The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
@mvstore_1081_h3
@mvstore_1096_h3
Tools
Tools
@mvstore_1082_p
@mvstore_1097_p
There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
@mvstore_1083_h3
@mvstore_1098_h3
Exception Handling
Exception Handling
@mvstore_1084_p
@mvstore_1099_p
This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur:
This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur:
@mvstore_1085_code
@mvstore_1100_code
IllegalStateException
IllegalStateException
@mvstore_1086_li
@mvstore_1101_li
if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
@mvstore_1087_code
@mvstore_1102_code
IllegalArgumentException
IllegalArgumentException
@mvstore_1088_li
@mvstore_1103_li
if a method was called with an illegal argument.
if a method was called with an illegal argument.
@mvstore_1089_code
@mvstore_1104_code
UnsupportedOperationException
UnsupportedOperationException
@mvstore_1090_li
@mvstore_1105_li
if a method was called that is not supported, for example trying to modify a read-only map or view.
if a method was called that is not supported, for example trying to modify a read-only map or view.
@mvstore_1091_code
@mvstore_1106_code
ConcurrentModificationException
ConcurrentModificationException
@mvstore_1092_li
@mvstore_1107_li
if the object is modified concurrently.
if the object is modified concurrently.
@mvstore_1093_h2
@mvstore_1108_h2
Similar Projects and Differences to Other Storage Engines
Similar Projects and Differences to Other Storage Engines
@mvstore_1094_p
@mvstore_1109_p
Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
@mvstore_1095_p
@mvstore_1110_p
The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
@mvstore_1096_p
@mvstore_1111_p
Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
@mvstore_1097_p
@mvstore_1112_p
The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
@mvstore_1098_h2
@mvstore_1113_h2
Current State
Current State
@mvstore_1099_p
@mvstore_1114_p
The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
@mvstore_1100_h2
@mvstore_1115_h2
Requirements
Requirements
@mvstore_1101_p
@mvstore_1116_p
The MVStore is included in the latest H2 jar file.
The MVStore is included in the latest H2 jar file.
@mvstore_1102_p
@mvstore_1117_p
There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
@mvstore_1103_p
@mvstore_1118_p
To build just the MVStore (without the database engine), run:
To build just the MVStore (without the database engine), run:
@mvstore_1104_p
@mvstore_1119_p
This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
#There is no limit for the following entities, except the memory and storage capacity: maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
#There is no limit for the following entities, except the memory and storage capacity: maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
@advanced_1403_li
@advanced_1403_li
#Querying from the metadata tables is slow if there are many tables (thousands).
@advanced_1404_li
#For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
#For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
@advanced_1404_h2
@advanced_1405_h2
用語集とリンク
用語集とリンク
@advanced_1405_th
@advanced_1406_th
用語
用語
@advanced_1406_th
@advanced_1407_th
説明
説明
@advanced_1407_td
@advanced_1408_td
AES-128
AES-128
@advanced_1408_td
@advanced_1409_td
#A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia: AES</a>
#A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia: AES</a>
@advanced_1409_td
@advanced_1410_td
Birthday Paradox
Birthday Paradox
@advanced_1410_td
@advanced_1411_td
#Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also: <a href="http://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia: Birthday Paradox</a>
#Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also: <a href="http://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia: Birthday Paradox</a>
@advanced_1411_td
@advanced_1412_td
Digest
Digest
@advanced_1412_td
@advanced_1413_td
#Protocol to protect a password (but not to protect data). See also: <a href="http://www.faqs.org/rfcs/rfc2617.html">RFC 2617: HTTP Digest Access Authentication</a>
#Protocol to protect a password (but not to protect data). See also: <a href="http://www.faqs.org/rfcs/rfc2617.html">RFC 2617: HTTP Digest Access Authentication</a>
@advanced_1413_td
@advanced_1414_td
GCJ
GCJ
@advanced_1414_td
@advanced_1415_td
#Compiler for Java. <a href="http://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href="http://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
#Compiler for Java. <a href="http://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href="http://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
@advanced_1415_td
@advanced_1416_td
HTTPS
HTTPS
@advanced_1416_td
@advanced_1417_td
#A protocol to provide security to HTTP connections. See also: <a href="http://www.ietf.org/rfc/rfc2818.txt">RFC 2818: HTTP Over TLS</a>
#A protocol to provide security to HTTP connections. See also: <a href="http://www.ietf.org/rfc/rfc2818.txt">RFC 2818: HTTP Over TLS</a>
@advanced_1417_td
@advanced_1418_td
Modes of Operation
Modes of Operation
@advanced_1418_a
@advanced_1419_a
#Wikipedia: Block cipher modes of operation
#Wikipedia: Block cipher modes of operation
@advanced_1419_td
@advanced_1420_td
Salt
Salt
@advanced_1420_td
@advanced_1421_td
#Random number to increase the security of passwords. See also: <a href="http://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia: Key derivation function</a>
#Random number to increase the security of passwords. See also: <a href="http://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia: Key derivation function</a>
#A security vulnerability where an application embeds SQL statements or expressions in user input. See also: <a href="http://en.wikipedia.org/wiki/SQL_injection">Wikipedia: SQL Injection</a>
#A security vulnerability where an application embeds SQL statements or expressions in user input. See also: <a href="http://en.wikipedia.org/wiki/SQL_injection">Wikipedia: SQL Injection</a>
@advanced_1425_td
@advanced_1426_td
Watermark Attack (透かし攻撃)
Watermark Attack (透かし攻撃)
@advanced_1426_td
@advanced_1427_td
#Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
#Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
@advanced_1427_td
@advanced_1428_td
SSL/TLS
SSL/TLS
@advanced_1428_td
@advanced_1429_td
#Secure Sockets Layer / Transport Layer Security. See also: <a href="http://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
#Secure Sockets Layer / Transport Layer Security. See also: <a href="http://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
@advanced_1429_td
@advanced_1430_td
XTEA
XTEA
@advanced_1430_td
@advanced_1431_td
#A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/XTEA">Wikipedia: XTEA</a>
#A block encryption algorithm. See also: <a href="http://en.wikipedia.org/wiki/XTEA">Wikipedia: XTEA</a>
@build_1000_h1
@build_1000_h1
...
@@ -3848,7 +3851,7 @@ jdbc:h2:mem:
...
@@ -3848,7 +3851,7 @@ jdbc:h2:mem:
接続が開始された時に他の設定を変更する
接続が開始された時に他の設定を変更する
@features_1389_p
@features_1389_p
# In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href="grammar.html">SQL Grammar</a>.
# In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href="grammar.html">SQL Grammar</a> or the <a href="../javadoc/org/h2/constant/DbSettings.html">DbSettings</a> javadoc.
@features_1390_h2
@features_1390_h2
カスタムファイル アクセスモード
カスタムファイル アクセスモード
...
@@ -6799,301 +6802,346 @@ H2 データベース エンジン
...
@@ -6799,301 +6802,346 @@ H2 データベース エンジン
@mvstore_1005_a
@mvstore_1005_a
# Features
# Features
@mvstore_1006_a
@mvstore_1006_div
# - <a href="#maps">Maps</a>
@mvstore_1007_div
# - <a href="#versions">Versions</a>
@mvstore_1008_div
# - <a href="#transactions">Transactions</a>
@mvstore_1009_div
# - <a href="#inMemory">In-Memory Performance and Usage</a>
@mvstore_1010_div
# - <a href="#dataTypes">Pluggable Data Types</a>
@mvstore_1011_div
# - <a href="#blob">BLOB Support</a>
@mvstore_1012_div
# - <a href="#pluggableMap">R-Tree and Pluggable Map Implementations</a>
@mvstore_1013_div
# - <a href="#caching">Concurrent Operations and Caching</a>
# Similar Projects and Differences to Other Storage Engines
# Similar Projects and Differences to Other Storage Engines
@mvstore_1007_a
@mvstore_1020_a
# Current State
# Current State
@mvstore_1008_a
@mvstore_1021_a
# Requirements
# Requirements
@mvstore_1009_h2
@mvstore_1022_h2
#Overview
#Overview
@mvstore_1010_p
@mvstore_1023_p
# The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
# The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
@mvstore_1011_li
@mvstore_1024_li
#MVStore stands for "multi-version store".
#MVStore stands for "multi-version store".
@mvstore_1012_li
@mvstore_1025_li
#Each store contains a number of maps (using the <code>java.util.Map</code> interface).
#Each store contains a number of maps (using the <code>java.util.Map</code> interface).
@mvstore_1013_li
@mvstore_1026_li
#Both file-based persistence and in-memory operation are supported.
#Both file-based persistence and in-memory operation are supported.
@mvstore_1014_li
@mvstore_1027_li
#It is intended to be fast, simple to use, and small.
#It is intended to be fast, simple to use, and small.
@mvstore_1015_li
@mvstore_1028_li
#Old versions of the data can be read concurrently with all other operations.
#Old versions of the data can be read concurrently with all other operations.
@mvstore_1016_li
@mvstore_1029_li
#Transaction are supported (currently only one transaction at a time).
#Transaction are supported.
@mvstore_1017_li
#Transactions (even if they are persisted) can be rolled back.
@mvstore_1018_li
@mvstore_1030_li
#The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
#The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
@mvstore_1019_h2
@mvstore_1031_h2
#Example Code
#Example Code
@mvstore_1020_p
@mvstore_1032_p
# The following sample code show how to create a store, open a map, add some data, and access the current and an old version:
# The following sample code show how to create a store, open a map, add some data, and access the current and an old version:
@mvstore_1021_h2
@mvstore_1033_h2
#Store Builder
#Store Builder
@mvstore_1022_p
@mvstore_1034_p
# The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options:
# The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options:
@mvstore_1023_li
@mvstore_1035_li
#cacheSizeMB: the cache size in MB.
#cacheSizeMB: the cache size in MB.
@mvstore_1024_li
@mvstore_1036_li
#compressData: compress the data when storing.
#compressData: compress the data when storing.
@mvstore_1025_li
@mvstore_1037_li
#encryptionKey: the encryption key for file encryption.
#encryptionKey: the encryption key for file encryption.
@mvstore_1026_li
@mvstore_1038_li
#fileName: the name of the file, for file based stores.
#fileName: the name of the file, for file based stores.
@mvstore_1027_li
@mvstore_1039_li
#readOnly: open the file in read-only mode.
#readOnly: open the file in read-only mode.
@mvstore_1028_li
@mvstore_1040_li
#writeBufferSize: the size of the write buffer in MB.
#writeBufferSize: the size of the write buffer in MB.
@mvstore_1029_li
@mvstore_1041_li
#writeDelay: the maximum delay until committed changes are stored (unless stored explicitly).
#writeDelay: the maximum delay until committed changes are stored (unless stored explicitly).
@mvstore_1030_h2
@mvstore_1042_h2
#R-Tree
#R-Tree
@mvstore_1031_p
@mvstore_1043_p
# The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows:
# The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows:
@mvstore_1032_p
@mvstore_1044_p
# The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
# The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
@mvstore_1033_h2
@mvstore_1045_h2
特徴
特徴
@mvstore_1034_h3
@mvstore_1046_h3
#Maps
#Maps
@mvstore_1035_p
@mvstore_1047_p
# Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
# Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
@mvstore_1036_p
@mvstore_1048_p
# Also supported, and very uncommon for maps, is fast index lookup: the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
# Also supported, and very uncommon for maps, is fast index lookup: the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
@mvstore_1037_p
@mvstore_1049_p
# In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
# In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
@mvstore_1038_h3
@mvstore_1050_h3
#Versions / Transactions
#Versions
@mvstore_1039_p
@mvstore_1051_p
# Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
# Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
@mvstore_1040_p
@mvstore_1052_p
# Versions / transactions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
# Versions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
@mvstore_1041_p
@mvstore_1053_p
# Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast: only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
# Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast: only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
@mvstore_1042_p
@mvstore_1054_p
# Rollback is supported (rollback to any old in-memory version or an old persisted version).
# Rollback is supported (rollback to any old in-memory version or an old persisted version).
@mvstore_1043_h3
@mvstore_1055_h3
#Transactions
@mvstore_1056_p
# The multi-version support is the basis for the transaction support. In the simple case, when only one transaction is open at a time, rolling back the transaction only requires to revert to an old version.
@mvstore_1057_p
# To support multiple concurrent open transactions, a transaction utility is included, the <code>TransactionStore</code>. This utility stores the changed entries in a separate map, similar to a transaction log (except that only the key of a changed row is stored, and the entries of a transaction are removed when the transaction is committed). The storage overhead of this utility is very small compared to the overhead of a regular transaction log. The tool supports PostgreSQL style "read committed" transaction isolation. There is no limit on the size of a transaction (the log is not kept in memory).
@mvstore_1058_h3
#In-Memory Performance and Usage
#In-Memory Performance and Usage
@mvstore_1044_p
@mvstore_1059_p
# Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
# Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
@mvstore_1045_p
@mvstore_1060_p
# The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
# The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
@mvstore_1046_p
@mvstore_1061_p
# If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
# If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
@mvstore_1047_h3
@mvstore_1062_h3
#Pluggable Data Types
#Pluggable Data Types
@mvstore_1048_p
@mvstore_1063_p
# Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported: <code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
# Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported: <code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
@mvstore_1049_p
@mvstore_1064_p
# Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
# Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
@mvstore_1050_p
@mvstore_1065_p
# The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
# The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
@mvstore_1051_h3
@mvstore_1066_h3
#BLOB Support
#BLOB Support
@mvstore_1052_p
@mvstore_1067_p
# There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
# There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
@mvstore_1053_h3
@mvstore_1068_h3
#R-Tree and Pluggable Map Implementations
#R-Tree and Pluggable Map Implementations
@mvstore_1054_p
@mvstore_1069_p
# The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
# The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
@mvstore_1055_h3
@mvstore_1070_h3
#Concurrent Operations and Caching
#Concurrent Operations and Caching
@mvstore_1056_p
@mvstore_1071_p
# The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
# The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
@mvstore_1057_p
@mvstore_1072_p
# Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
# Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
@mvstore_1058_p
@mvstore_1073_p
# Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
# Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
@mvstore_1059_p
@mvstore_1074_p
# The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
# The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
@mvstore_1060_p
@mvstore_1075_p
# With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
# With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
@mvstore_1061_p
@mvstore_1076_p
# For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
# For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
@mvstore_1062_h3
@mvstore_1077_h3
#Log Structured Storage
#Log Structured Storage
@mvstore_1063_p
@mvstore_1078_p
# Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
# Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
@mvstore_1064_p
@mvstore_1079_p
# When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index: all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
# When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index: all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
@mvstore_1065_p
@mvstore_1080_p
# There are usually two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
# There are usually two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
@mvstore_1066_p
@mvstore_1081_p
# There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
# There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
@mvstore_1067_p
@mvstore_1082_p
# Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
# Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
@mvstore_1068_p
@mvstore_1083_p
# Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
# Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
@mvstore_1069_h3
@mvstore_1084_h3
#File System Abstraction, File Locking and Online Backup
#File System Abstraction, File Locking and Online Backup
@mvstore_1070_p
@mvstore_1085_p
# The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
# The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
@mvstore_1071_p
@mvstore_1086_p
# Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
# Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
@mvstore_1072_p
@mvstore_1087_p
# The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
# The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
@mvstore_1073_h3
@mvstore_1088_h3
#Encrypted Files
#Encrypted Files
@mvstore_1074_p
@mvstore_1089_p
# File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows:
# File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows:
@mvstore_1075_p
@mvstore_1090_p
# The following algorithms and settings are used:
# The following algorithms and settings are used:
@mvstore_1076_li
@mvstore_1091_li
#The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
#The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
@mvstore_1077_li
@mvstore_1092_li
#The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
#The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
@mvstore_1078_li
@mvstore_1093_li
#The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
#The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
@mvstore_1079_li
@mvstore_1094_li
#To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
#To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
@mvstore_1080_li
@mvstore_1095_li
#The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
#The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
@mvstore_1081_h3
@mvstore_1096_h3
#Tools
#Tools
@mvstore_1082_p
@mvstore_1097_p
# There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
# There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
@mvstore_1083_h3
@mvstore_1098_h3
#Exception Handling
#Exception Handling
@mvstore_1084_p
@mvstore_1099_p
# This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur:
# This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur:
@mvstore_1085_code
@mvstore_1100_code
#IllegalStateException
#IllegalStateException
@mvstore_1086_li
@mvstore_1101_li
# if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
# if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
@mvstore_1087_code
@mvstore_1102_code
#IllegalArgumentException
#IllegalArgumentException
@mvstore_1088_li
@mvstore_1103_li
# if a method was called with an illegal argument.
# if a method was called with an illegal argument.
@mvstore_1089_code
@mvstore_1104_code
#UnsupportedOperationException
#UnsupportedOperationException
@mvstore_1090_li
@mvstore_1105_li
# if a method was called that is not supported, for example trying to modify a read-only map or view.
# if a method was called that is not supported, for example trying to modify a read-only map or view.
@mvstore_1091_code
@mvstore_1106_code
#ConcurrentModificationException
#ConcurrentModificationException
@mvstore_1092_li
@mvstore_1107_li
# if the object is modified concurrently.
# if the object is modified concurrently.
@mvstore_1093_h2
@mvstore_1108_h2
#Similar Projects and Differences to Other Storage Engines
#Similar Projects and Differences to Other Storage Engines
@mvstore_1094_p
@mvstore_1109_p
# Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
# Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
@mvstore_1095_p
@mvstore_1110_p
# The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
# The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
@mvstore_1096_p
@mvstore_1111_p
# Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
# Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
@mvstore_1097_p
@mvstore_1112_p
# The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
# The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
@mvstore_1098_h2
@mvstore_1113_h2
#Current State
#Current State
@mvstore_1099_p
@mvstore_1114_p
# The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
# The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
@mvstore_1100_h2
@mvstore_1115_h2
必要条件
必要条件
@mvstore_1101_p
@mvstore_1116_p
# The MVStore is included in the latest H2 jar file.
# The MVStore is included in the latest H2 jar file.
@mvstore_1102_p
@mvstore_1117_p
# There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
# There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
@mvstore_1103_p
@mvstore_1118_p
# To build just the MVStore (without the database engine), run:
# To build just the MVStore (without the database engine), run:
@mvstore_1104_p
@mvstore_1119_p
# This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
# This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
@@ -401,34 +401,35 @@ advanced_1399_li=The maximum number of rows per table is 2^64.
...
@@ -401,34 +401,35 @@ advanced_1399_li=The maximum number of rows per table is 2^64.
advanced_1400_li=Main memory requirements\:The larger the database, the more main memory is required. With the version 1.1 storage mechanism, the minimum main memory required for a 12 GB database was around 240 MB. With the current page store, the minimum main memory required is much lower, around 1 MB for each 8 GB database file size.
advanced_1400_li=Main memory requirements\:The larger the database, the more main memory is required. With the version 1.1 storage mechanism, the minimum main memory required for a 12 GB database was around 240 MB. With the current page store, the minimum main memory required is much lower, around 1 MB for each 8 GB database file size.
advanced_1401_li=Limit on the complexity of SQL statements. Statements of the following form will result in a stack overflow exception\:
advanced_1401_li=Limit on the complexity of SQL statements. Statements of the following form will result in a stack overflow exception\:
advanced_1402_li=There is no limit for the following entities, except the memory and storage capacity\:maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
advanced_1402_li=There is no limit for the following entities, except the memory and storage capacity\:maximum identifier length (table name, column name, and so on); maximum number of tables, columns, indexes, triggers, and other database objects; maximum statement length, number of parameters per statement, tables per statement, expressions in order by, group by, having, and so on; maximum rows per query; maximum columns per table, columns per index, indexes per table, lob columns per table, and so on; maximum row length, index row length, select row length; maximum length of a varchar column, decimal column, literal in a statement.
advanced_1403_li=For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
advanced_1403_li=Querying from the metadata tables is slow if there are many tables (thousands).
advanced_1404_h2=Glossary and Links
advanced_1404_li=For limitations on data types, see the documentation of the respective Java data type or the data type documentation of this database.
advanced_1405_th=Term
advanced_1405_h2=Glossary and Links
advanced_1406_th=Description
advanced_1406_th=Term
advanced_1407_td=AES-128
advanced_1407_th=Description
advanced_1408_td=A block encryption algorithm. See also\:<a href\="http\://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia\:AES</a>
advanced_1408_td=AES-128
advanced_1409_td=Birthday Paradox
advanced_1409_td=A block encryption algorithm. See also\:<a href\="http\://en.wikipedia.org/wiki/Advanced_Encryption_Standard">Wikipedia\:AES</a>
advanced_1410_td=Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also\:<a href\="http\://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia\:Birthday Paradox</a>
advanced_1410_td=Birthday Paradox
advanced_1411_td=Digest
advanced_1411_td=Describes the higher than expected probability that two persons in a room have the same birthday. Also valid for randomly generated UUIDs. See also\:<a href\="http\://en.wikipedia.org/wiki/Birthday_paradox">Wikipedia\:Birthday Paradox</a>
advanced_1412_td=Protocol to protect a password (but not to protect data). See also\:<a href\="http\://www.faqs.org/rfcs/rfc2617.html">RFC 2617\:HTTP Digest Access Authentication</a>
advanced_1412_td=Digest
advanced_1413_td=GCJ
advanced_1413_td=Protocol to protect a password (but not to protect data). See also\:<a href\="http\://www.faqs.org/rfcs/rfc2617.html">RFC 2617\:HTTP Digest Access Authentication</a>
advanced_1414_td=Compiler for Java. <a href\="http\://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href\="http\://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
advanced_1414_td=GCJ
advanced_1415_td=HTTPS
advanced_1415_td=Compiler for Java. <a href\="http\://gcc.gnu.org/java">GNU Compiler for the Java</a> and <a href\="http\://www.dobysoft.com/products/nativej">NativeJ (commercial)</a>
advanced_1416_td=A protocol to provide security to HTTP connections. See also\:<a href\="http\://www.ietf.org/rfc/rfc2818.txt">RFC 2818\:HTTP Over TLS</a>
advanced_1416_td=HTTPS
advanced_1417_td=Modes of Operation
advanced_1417_td=A protocol to provide security to HTTP connections. See also\:<a href\="http\://www.ietf.org/rfc/rfc2818.txt">RFC 2818\:HTTP Over TLS</a>
advanced_1418_a=Wikipedia\:Block cipher modes of operation
advanced_1418_td=Modes of Operation
advanced_1419_td=Salt
advanced_1419_a=Wikipedia\:Block cipher modes of operation
advanced_1420_td=Random number to increase the security of passwords. See also\:<a href\="http\://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia\:Key derivation function</a>
advanced_1420_td=Salt
advanced_1421_td=SHA-256
advanced_1421_td=Random number to increase the security of passwords. See also\:<a href\="http\://en.wikipedia.org/wiki/Key_derivation_function">Wikipedia\:Key derivation function</a>
advanced_1422_td=A cryptographic one-way hash function. See also\:<a href\="http\://en.wikipedia.org/wiki/SHA_family">Wikipedia\:SHA hash functions</a>
advanced_1422_td=SHA-256
advanced_1423_td=SQL Injection
advanced_1423_td=A cryptographic one-way hash function. See also\:<a href\="http\://en.wikipedia.org/wiki/SHA_family">Wikipedia\:SHA hash functions</a>
advanced_1424_td=A security vulnerability where an application embeds SQL statements or expressions in user input. See also\:<a href\="http\://en.wikipedia.org/wiki/SQL_injection">Wikipedia\:SQL Injection</a>
advanced_1424_td=SQL Injection
advanced_1425_td=Watermark Attack
advanced_1425_td=A security vulnerability where an application embeds SQL statements or expressions in user input. See also\:<a href\="http\://en.wikipedia.org/wiki/SQL_injection">Wikipedia\:SQL Injection</a>
advanced_1426_td=Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
advanced_1426_td=Watermark Attack
advanced_1427_td=SSL/TLS
advanced_1427_td=Security problem of certain encryption programs where the existence of certain data can be proven without decrypting. For more information, search in the internet for 'watermark attack cryptoloop'
advanced_1428_td=Secure Sockets Layer / Transport Layer Security. See also\:<a href\="http\://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
advanced_1428_td=SSL/TLS
advanced_1429_td=XTEA
advanced_1429_td=Secure Sockets Layer / Transport Layer Security. See also\:<a href\="http\://java.sun.com/products/jsse/">Java Secure Socket Extension (JSSE)</a>
advanced_1430_td=A block encryption algorithm. See also\:<a href\="http\://en.wikipedia.org/wiki/XTEA">Wikipedia\:XTEA</a>
advanced_1430_td=XTEA
advanced_1431_td=A block encryption algorithm. See also\:<a href\="http\://en.wikipedia.org/wiki/XTEA">Wikipedia\:XTEA</a>
build_1000_h1=Build
build_1000_h1=Build
build_1001_a=\ Portability
build_1001_a=\ Portability
build_1002_a=\ Environment
build_1002_a=\ Environment
...
@@ -1281,7 +1282,7 @@ features_1385_p=\ Backslashes within the init script (for example within a runsc
...
@@ -1281,7 +1282,7 @@ features_1385_p=\ Backslashes within the init script (for example within a runsc
features_1386_h2=Ignore Unknown Settings
features_1386_h2=Ignore Unknown Settings
features_1387_p=\ Some applications (for example OpenOffice.org Base) pass some additional parameters when connecting to the database. Why those parameters are passed is unknown. The parameters <code>PREFERDOSLIKELINEENDS</code> and <code>IGNOREDRIVERPRIVILEGES</code> are such examples; they are simply ignored to improve the compatibility with OpenOffice.org. If an application passes other parameters when connecting to the database, usually the database throws an exception saying the parameter is not supported. It is possible to ignored such parameters by adding <code>;IGNORE_UNKNOWN_SETTINGS\=TRUE</code> to the database URL.
features_1387_p=\ Some applications (for example OpenOffice.org Base) pass some additional parameters when connecting to the database. Why those parameters are passed is unknown. The parameters <code>PREFERDOSLIKELINEENDS</code> and <code>IGNOREDRIVERPRIVILEGES</code> are such examples; they are simply ignored to improve the compatibility with OpenOffice.org. If an application passes other parameters when connecting to the database, usually the database throws an exception saying the parameter is not supported. It is possible to ignored such parameters by adding <code>;IGNORE_UNKNOWN_SETTINGS\=TRUE</code> to the database URL.
features_1388_h2=Changing Other Settings when Opening a Connection
features_1388_h2=Changing Other Settings when Opening a Connection
features_1389_p=\ In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting\=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href\="grammar.html">SQL Grammar</a>.
features_1389_p=\ In addition to the settings already described, other database settings can be passed in the database URL. Adding <code>;setting\=value</code> at the end of a database URL is the same as executing the statement <code>SET setting value</code> just after connecting. For a list of supported settings, see <a href\="grammar.html">SQL Grammar</a> or the <a href\="../javadoc/org/h2/constant/DbSettings.html">DbSettings</a> javadoc.
features_1390_h2=Custom File Access Mode
features_1390_h2=Custom File Access Mode
features_1391_p=\ Usually, the database opens the database file with the access mode <code>rw</code>, meaning read-write (except for read only databases, where the mode <code>r</code> is used). To open a database in read-only mode if the database file is not read-only, use <code>ACCESS_MODE_DATA\=r</code>. Also supported are <code>rws</code> and <code>rwd</code>. This setting must be specified in the database URL\:
features_1391_p=\ Usually, the database opens the database file with the access mode <code>rw</code>, meaning read-write (except for read only databases, where the mode <code>r</code> is used). To open a database in read-only mode if the database file is not read-only, use <code>ACCESS_MODE_DATA\=r</code>. Also supported are <code>rws</code> and <code>rwd</code>. This setting must be specified in the database URL\:
features_1392_p=\ For more information see <a href\="advanced.html\#durability_problems">Durability Problems</a>. On many operating systems the access mode <code>rws</code> does not guarantee that the data is written to the disk.
features_1392_p=\ For more information see <a href\="advanced.html\#durability_problems">Durability Problems</a>. On many operating systems the access mode <code>rws</code> does not guarantee that the data is written to the disk.
...
@@ -2265,105 +2266,120 @@ mvstore_1002_a=\ Example Code
...
@@ -2265,105 +2266,120 @@ mvstore_1002_a=\ Example Code
mvstore_1003_a=\ Store Builder
mvstore_1003_a=\ Store Builder
mvstore_1004_a=\ R-Tree
mvstore_1004_a=\ R-Tree
mvstore_1005_a=\ Features
mvstore_1005_a=\ Features
mvstore_1006_a=\ Similar Projects and Differences to Other Storage Engines
mvstore_1009_div=\ - <a href\="\#inMemory">In-Memory Performance and Usage</a>
mvstore_1010_p=\ The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
mvstore_1010_div=\ - <a href\="\#dataTypes">Pluggable Data Types</a>
mvstore_1011_li=MVStore stands for "multi-version store".
mvstore_1017_li=Transactions (even if they are persisted) can be rolled back.
mvstore_1017_div=\ - <a href\="\#tools">Tools</a>
mvstore_1018_li=The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
mvstore_1019_a=\ Similar Projects and Differences to Other Storage Engines
mvstore_1020_p=\ The following sample code show how to create a store, open a map, add some data, and access the current and an old version\:
mvstore_1020_a=\ Current State
mvstore_1021_h2=Store Builder
mvstore_1021_a=\ Requirements
mvstore_1022_p=\ The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options\:
mvstore_1022_h2=Overview
mvstore_1023_li=cacheSizeMB\:the cache size in MB.
mvstore_1023_p=\ The MVStore is work in progress, and is planned to be the next storage subsystem of H2. But it can be also directly within an application, without using JDBC or SQL.
mvstore_1024_li=compressData\:compress the data when storing.
mvstore_1024_li=MVStore stands for "multi-version store".
mvstore_1025_li=encryptionKey\:the encryption key for file encryption.
mvstore_1025_li=Each store contains a number of maps (using the <code>java.util.Map</code> interface).
mvstore_1026_li=fileName\:the name of the file, for file based stores.
mvstore_1026_li=Both file-based persistence and in-memory operation are supported.
mvstore_1027_li=readOnly\:open the file in read-only mode.
mvstore_1027_li=It is intended to be fast, simple to use, and small.
mvstore_1028_li=writeBufferSize\:the size of the write buffer in MB.
mvstore_1028_li=Old versions of the data can be read concurrently with all other operations.
mvstore_1029_li=writeDelay\:the maximum delay until committed changes are stored (unless stored explicitly).
mvstore_1029_li=Transaction are supported.
mvstore_1030_h2=R-Tree
mvstore_1030_li=The tool is very modular. It supports pluggable data types / serialization, pluggable map implementations (B-tree, R-tree, concurrent B-tree currently), BLOB storage, and a file system abstraction to support encrypted files and zip files.
mvstore_1031_p=\ The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows\:
mvstore_1031_h2=Example Code
mvstore_1032_p=\ The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
mvstore_1032_p=\ The following sample code show how to create a store, open a map, add some data, and access the current and an old version\:
mvstore_1033_h2=Features
mvstore_1033_h2=Store Builder
mvstore_1034_h3=Maps
mvstore_1034_p=\ The <code>MVStore.Builder</code> provides a fluid interface to build a store if more complex configuration options are used. The following code contains all supported configuration options\:
mvstore_1035_p=\ Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
mvstore_1035_li=cacheSizeMB\:the cache size in MB.
mvstore_1036_p=\ Also supported, and very uncommon for maps, is fast index lookup\:the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
mvstore_1036_li=compressData\:compress the data when storing.
mvstore_1037_p=\ In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
mvstore_1037_li=encryptionKey\:the encryption key for file encryption.
mvstore_1038_h3=Versions / Transactions
mvstore_1038_li=fileName\:the name of the file, for file based stores.
mvstore_1039_p=\ Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
mvstore_1039_li=readOnly\:open the file in read-only mode.
mvstore_1040_p=\ Versions / transactions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
mvstore_1040_li=writeBufferSize\:the size of the write buffer in MB.
mvstore_1041_p=\ Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast\:only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
mvstore_1041_li=writeDelay\:the maximum delay until committed changes are stored (unless stored explicitly).
mvstore_1042_p=\ Rollback is supported (rollback to any old in-memory version or an old persisted version).
mvstore_1042_h2=R-Tree
mvstore_1043_h3=In-Memory Performance and Usage
mvstore_1043_p=\ The <code>MVRTreeMap</code> is an R-tree implementation that supports fast spatial queries. It can be used as follows\:
mvstore_1044_p=\ Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
mvstore_1044_p=\ The default number of dimensions is 2. To use a different number of dimensions, call <code>new MVRTreeMap.Builder<String>().dimensions(3)</code>. The minimum number of dimensions is 1, the maximum is 255.
mvstore_1045_p=\ The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
mvstore_1045_h2=Features
mvstore_1046_p=\ If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
mvstore_1046_h3=Maps
mvstore_1047_h3=Pluggable Data Types
mvstore_1047_p=\ Each store supports a set of named maps. A map is sorted by key, and supports the common lookup operations, including access to the first and last key, iterate over some or all keys, and so on.
mvstore_1048_p=\ Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported\:<code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
mvstore_1048_p=\ Also supported, and very uncommon for maps, is fast index lookup\:the keys of the map can be accessed like a list (get the key at the given index, get the index of a certain key). That means getting the median of two keys is trivial, and range of keys can be counted very quickly. The iterator supports fast skipping. This is possible because internally, each map is organized in the form of a counted B+-tree.
mvstore_1049_p=\ Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
mvstore_1049_p=\ In database terms, a map can be used like a table, where the key of the map is the primary key of the table, and the value is the row. A map can also represent an index, where the key of the map is the key of the index, and the value of the map is the primary key of the table (for non-unique indexes, the key of the map must also contain the primary key).
mvstore_1050_p=\ The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
mvstore_1050_h3=Versions
mvstore_1051_h3=BLOB Support
mvstore_1051_p=\ Multiple versions are supported. A version is a snapshot of all the data of all maps at a given point in time. A transaction is a number of actions between two versions.
mvstore_1052_p=\ There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
mvstore_1052_p=\ Versions are not immediately persisted; instead, only the version counter is incremented. If there is a change after switching to a new version, a snapshot of the old version is kept in memory, so that it can still be read.
mvstore_1053_h3=R-Tree and Pluggable Map Implementations
mvstore_1053_p=\ Old persisted versions are readable until the old data was explicitly overwritten. Creating a snapshot is fast\:only the pages that are changed after a snapshot are copied. This behavior is also called COW (copy on write).
mvstore_1054_p=\ The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
mvstore_1054_p=\ Rollback is supported (rollback to any old in-memory version or an old persisted version).
mvstore_1055_h3=Concurrent Operations and Caching
mvstore_1055_h3=Transactions
mvstore_1056_p=\ The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
mvstore_1056_p=\ The multi-version support is the basis for the transaction support. In the simple case, when only one transaction is open at a time, rolling back the transaction only requires to revert to an old version.
mvstore_1057_p=\ Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
mvstore_1057_p=\ To support multiple concurrent open transactions, a transaction utility is included, the <code>TransactionStore</code>. This utility stores the changed entries in a separate map, similar to a transaction log (except that only the key of a changed row is stored, and the entries of a transaction are removed when the transaction is committed). The storage overhead of this utility is very small compared to the overhead of a regular transaction log. The tool supports PostgreSQL style "read committed" transaction isolation. There is no limit on the size of a transaction (the log is not kept in memory).
mvstore_1058_p=\ Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
mvstore_1058_h3=In-Memory Performance and Usage
mvstore_1059_p=\ The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
mvstore_1059_p=\ Performance of in-memory operations is comparable with <code>java.util.TreeMap</code> (many operations are actually faster), but usually slower than <code>java.util.HashMap</code>.
mvstore_1060_p=\ With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
mvstore_1060_p=\ The memory overhead for large maps is slightly better than for the regular map implementations, but there is a higher overhead per map. For maps with less than 25 entries, the regular map implementations use less memory on average.
mvstore_1061_p=\ For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
mvstore_1061_p=\ If no file name is specified, the store operates purely in memory. Except for persisting data, all features are supported in this mode (multi-versioning, index lookup, R-tree and so on). If a file name is specified, all operations occur in memory (with the same performance characteristics) until data is persisted.
mvstore_1062_h3=Log Structured Storage
mvstore_1062_h3=Pluggable Data Types
mvstore_1063_p=\ Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
mvstore_1063_p=\ Serialization is pluggable. The default serialization currently supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported\:<code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal, String, UUID, Date</code> and arrays (both primitive arrays and object arrays).
mvstore_1064_p=\ When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index\:all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
mvstore_1064_p=\ Parameterized data types are supported (for example one could build a string data type that limits the length for some reason).
mvstore_1065_p=\ There are usually two write operations per chunk\:one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
mvstore_1065_p=\ The storage engine itself does not have any length limits, so that keys, values, pages, and chunks can be very big (as big as fits in memory). Also, there is no inherent limit to the number of maps and chunks. Due to using a log structured storage, there is no special case handling for large keys or pages.
mvstore_1066_p=\ There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
mvstore_1066_h3=BLOB Support
mvstore_1067_p=\ Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
mvstore_1067_p=\ There is a mechanism that stores large binary objects by splitting them into smaller blocks. This allows to store objects that don't fit in memory. Streaming as well as random access reads on such objects are supported. This tool is written on top of the store (only using the map interface).
mvstore_1068_p=\ Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
mvstore_1068_h3=R-Tree and Pluggable Map Implementations
mvstore_1069_h3=File System Abstraction, File Locking and Online Backup
mvstore_1069_p=\ The map implementation is pluggable. In addition to the default <code>MVMap</code> (multi-version map), there is a multi-version R-tree map implementation for spatial operations (contain and intersection; nearest neighbor is not yet implemented).
mvstore_1070_p=\ The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
mvstore_1070_h3=Concurrent Operations and Caching
mvstore_1071_p=\ Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
mvstore_1071_p=\ The default map implementation supports concurrent reads on old versions of the data. All such read operations can occur in parallel. Concurrent reads from the page cache, as well as concurrent reads from the file system are supported.
mvstore_1072_p=\ The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
mvstore_1072_p=\ Storing changes can occur concurrently to modifying the data, as it operates on a snapshot.
mvstore_1073_h3=Encrypted Files
mvstore_1073_p=\ Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
mvstore_1074_p=\ File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows\:
mvstore_1074_p=\ The default map implementation does not support concurrent modification operations on a map (the same as <code>HashMap</code> and <code>TreeMap</code>). Similar to those classes, the map tries to detect concurrent modification.
mvstore_1075_p=\ The following algorithms and settings are used\:
mvstore_1075_p=\ With the <code>MVMapConcurrent</code> implementation, read operations even on the newest version can happen concurrently with all other operations, without risk of corruption. This comes with slightly reduced speed in single threaded mode, the same as with other <code>ConcurrentHashMap</code> implementations. Write operations first read the relevant area from disk to memory (this can happen concurrently), and only then modify the data. The in-memory part of write operations is synchronized.
mvstore_1076_li=The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
mvstore_1076_p=\ For fully scalable concurrent write operations to a map (in-memory and to disk), the map could be split into multiple maps in different stores ('sharding'). The plan is to add such a mechanism later when needed.
mvstore_1077_li=The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
mvstore_1077_h3=Log Structured Storage
mvstore_1078_li=The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
mvstore_1078_p=\ Changes are buffered in memory, and once enough changes have accumulated, they are written in one continuous disk write operation. (According to a test, write throughput of a common SSD gets higher the larger the block size, until a block size of 2 MB, and then does not further increase.) By default, committed changes are automatically written once every second in a background thread, even if only little data was changed. Changes can also be written explicitly by calling <code>store()</code>. To avoid out of memory, uncommitted changes are also written when needed, however they are rolled back when closing the store, or at the latest (when the store was not correctly closed) when opening the store.
mvstore_1079_li=To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
mvstore_1079_p=\ When storing, all changed pages are serialized, optionally compressed using the LZF algorithm, and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read this version of the data). There is no separate index\:all data is stored as a list of pages. Per store, there is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
mvstore_1080_li=The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
mvstore_1080_p=\ There are usually two write operations per chunk\:one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk). If the chunk is appended at the end of the file, the file header is only written at the end of the chunk.
mvstore_1081_h3=Tools
mvstore_1081_p=\ There is no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten by default).
mvstore_1082_p=\ There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
mvstore_1082_p=\ Old data is kept for at least 45 seconds (configurable), so that there are no explicit sync operations required to guarantee data consistency, but an application can also sync explicitly when needed. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
mvstore_1083_h3=Exception Handling
mvstore_1083_p=\ Compared to traditional storage engines (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
mvstore_1084_p=\ This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur\:
mvstore_1084_h3=File System Abstraction, File Locking and Online Backup
mvstore_1085_code=IllegalStateException
mvstore_1085_p=\ The file system is pluggable (the same file system abstraction is used as H2 uses). The file can be encrypted using an encrypting file system. Other file system implementations support reading from a compressed zip or jar file.
mvstore_1086_li=\ if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
mvstore_1086_p=\ Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
mvstore_1087_code=IllegalArgumentException
mvstore_1087_p=\ The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
mvstore_1088_li=\ if a method was called with an illegal argument.
mvstore_1088_h3=Encrypted Files
mvstore_1089_code=UnsupportedOperationException
mvstore_1089_p=\ File encryption ensures the data can only be read with the correct password. Data can be encrypted as follows\:
mvstore_1090_li=\ if a method was called that is not supported, for example trying to modify a read-only map or view.
mvstore_1090_p=\ The following algorithms and settings are used\:
mvstore_1091_code=ConcurrentModificationException
mvstore_1091_li=The password char array is cleared after use, to reduce the risk that the password is stolen even if the attacker has access to the main memory.
mvstore_1092_li=\ if the object is modified concurrently.
mvstore_1092_li=The password is hashed according to the PBKDF2 standard, using the SHA-256 hash algorithm.
mvstore_1093_h2=Similar Projects and Differences to Other Storage Engines
mvstore_1093_li=The length of the salt is 64 bits, so that an attacker can not use a pre-calculated password hash table (rainbow table). It is generated using a cryptographically secure random number generator.
mvstore_1094_p=\ Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
mvstore_1094_li=To speed up opening an encrypted stores on Android, the number of PBKDF2 iterations is 10. The higher the value, the better the protection against brute-force password cracking attacks, but the slower is opening a file.
mvstore_1095_p=\ The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
mvstore_1095_li=The file itself is encrypted using the standardized disk encryption mode XTS-AES. Only little more than one AES-128 round per block is needed.
mvstore_1096_p=\ Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
mvstore_1096_h3=Tools
mvstore_1097_p=\ The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
mvstore_1097_p=\ There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
mvstore_1098_h2=Current State
mvstore_1098_h3=Exception Handling
mvstore_1099_p=\ The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
mvstore_1099_p=\ This tool does not throw checked exceptions. Instead, unchecked exceptions are thrown if needed. The error message always contains the version of the tool. The following exceptions can occur\:
mvstore_1100_h2=Requirements
mvstore_1100_code=IllegalStateException
mvstore_1101_p=\ The MVStore is included in the latest H2 jar file.
mvstore_1101_li=\ if a map was already closed or an IO exception occurred, for example if the file was locked, is already closed, could not be opened or closed, if reading or writing failed, if the file is corrupt, or if there is an internal error in the tool.
mvstore_1102_p=\ There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
mvstore_1102_code=IllegalArgumentException
mvstore_1103_p=\ To build just the MVStore (without the database engine), run\:
mvstore_1103_li=\ if a method was called with an illegal argument.
mvstore_1104_p=\ This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
mvstore_1104_code=UnsupportedOperationException
mvstore_1105_li=\ if a method was called that is not supported, for example trying to modify a read-only map or view.
mvstore_1106_code=ConcurrentModificationException
mvstore_1107_li=\ if the object is modified concurrently.
mvstore_1108_h2=Similar Projects and Differences to Other Storage Engines
mvstore_1109_p=\ Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java and Android application.
mvstore_1110_p=\ The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
mvstore_1111_p=\ Like SQLite, the MVStore keeps all data in one file. Unlike SQLite, the MVStore uses is a log structured storage. The plan is to make the MVStore both easier to use as well as faster than SQLite. In a recent (very simple) test, the MVStore was about twice as fast as SQLite on Android.
mvstore_1112_p=\ The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage. The MVStore does not have a record size limit.
mvstore_1113_h2=Current State
mvstore_1114_p=\ The code is still experimental at this stage. The API as well as the behavior may partially change. Features may be added and removed (even thought the main features will stay).
mvstore_1115_h2=Requirements
mvstore_1116_p=\ The MVStore is included in the latest H2 jar file.
mvstore_1117_p=\ There are no special requirements to use it. The MVStore should run on any JVM as well as on Android.
mvstore_1118_p=\ To build just the MVStore (without the database engine), run\:
mvstore_1119_p=\ This will create the file <code>bin/h2mvstore-1.3.170.jar</code> (about 130 KB).
