@@ -6872,78 +6872,81 @@ Concurrent Operations and Caching
Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
@mvstore_1049_p
Concurrent modification operations on the maps are currently not supported, however it is planned to support an additional map implementation that supports concurrent writes (at the cost of speed if used in a single thread, same as <code>ConcurrentHashMap</code>).
Concurrent modification operations on a map are currently not supported (the same as <code>HashMap</code> and <code>TreeMap</code>), however it is planned to support an additional map implementation that supports concurrent writes (at the cost of speed if used in a single thread, same as <code>ConcurrentHashMap</code>).
@mvstore_1050_h3
@mvstore_1050_p
Storing changes can occur concurrently to modifying the data, as <code>store()</code> operates on a snapshot.
@mvstore_1051_h3
Log Structured Storage
@mvstore_1051_p
@mvstore_1052_p
Currently, <code>store()</code> needs to be called explicitly to save changes. Changes are buffered in memory, and once enough changes have accumulated (for example 2 MB), all changes are written in one continuous disk write operation. But of course, if needed, changes can also be persisted if only little data was changed. The estimated amount of unsaved changes is tracked. The plan is to automatically store in a background thread once there are enough changes.
@mvstore_1052_p
@mvstore_1053_p
When storing, all changed pages are serialized, compressed using the LZF algorithm (this can be disabled), and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read old data). There is no separate index: all data is stored as a list of pages. Per store, the is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
@mvstore_1053_p
@mvstore_1054_p
There are currently two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk), but the plan is to write the file header only once in a while, in a way that still allows to open a store very quickly.
@mvstore_1054_p
@mvstore_1055_p
There is currently no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten). To efficiently persist very small transactions, the plan is to support a transaction log where only the deltas is stored, until enough changes have accumulated to persist a chunk. Old versions are kept and are readable until they are no longer needed.
@mvstore_1055_p
@mvstore_1056_p
The plan is to keep all old data for at least one or two minutes (configurable), so that there are no explicit sync operations required to guarantee data consistency. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
@mvstore_1056_p
@mvstore_1057_p
Compared to regular databases (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
@mvstore_1057_h3
@mvstore_1058_h3
File System Abstraction, File Locking and Online Backup
@mvstore_1058_p
@mvstore_1059_p
The file system is pluggable (the same file system abstraction is used as H2 uses). Support for encryption is planned using an encrypting file system. Other file system implementations support reading from a compressed zip or tar file.
@mvstore_1059_p
@mvstore_1060_p
Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
@mvstore_1060_p
@mvstore_1061_p
The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
@mvstore_1061_h3
@mvstore_1062_h3
Tools
@mvstore_1062_p
@mvstore_1063_p
There is a builder for store instances (<code>MVStoreBuilder</code>) with a fluent API to simplify building a store instance.
@mvstore_1063_p
@mvstore_1064_p
There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
@mvstore_1064_h2
@mvstore_1065_h2
Similar Projects and Differences to Other Storage Engines
@mvstore_1065_p
@mvstore_1066_p
Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java application.
@mvstore_1066_p
@mvstore_1067_p
The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
@mvstore_1067_p
@mvstore_1068_p
Like SQLite, the MVStore keeps all data in one file. The plan is to make the MVStore easier to use and faster than SQLite on Android (this was not recently tested, however an initial test was successful).
@mvstore_1068_p
@mvstore_1069_p
The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage.
@mvstore_1069_h2
@mvstore_1070_h2
Current State
@mvstore_1070_p
@mvstore_1071_p
The code is still very experimental at this stage. The API as well as the behavior will probably change. Features may be added and removed (even thought the main features will stay).
@mvstore_1071_h2
@mvstore_1072_h2
Requirements
@mvstore_1072_p
@mvstore_1073_p
The MVStore is included in the latest H2 jar file.
@mvstore_1073_p
@mvstore_1074_p
There are no special requirements to use it. The MVStore should run on any JVM as well as on Android (even thought this was not tested recently).
# Caching is done on the page level. The page cache is a concurrent LIRS cache, which should be resistant against scan operations.
@mvstore_1049_p
# Concurrent modification operations on the maps are currently not supported, however it is planned to support an additional map implementation that supports concurrent writes (at the cost of speed if used in a single thread, same as <code>ConcurrentHashMap</code>).
# Concurrent modification operations on a map are currently not supported (the same as <code>HashMap</code> and <code>TreeMap</code>), however it is planned to support an additional map implementation that supports concurrent writes (at the cost of speed if used in a single thread, same as <code>ConcurrentHashMap</code>).
@mvstore_1050_h3
@mvstore_1050_p
# Storing changes can occur concurrently to modifying the data, as <code>store()</code> operates on a snapshot.
@mvstore_1051_h3
#Log Structured Storage
@mvstore_1051_p
@mvstore_1052_p
# Currently, <code>store()</code> needs to be called explicitly to save changes. Changes are buffered in memory, and once enough changes have accumulated (for example 2 MB), all changes are written in one continuous disk write operation. But of course, if needed, changes can also be persisted if only little data was changed. The estimated amount of unsaved changes is tracked. The plan is to automatically store in a background thread once there are enough changes.
@mvstore_1052_p
@mvstore_1053_p
# When storing, all changed pages are serialized, compressed using the LZF algorithm (this can be disabled), and written sequentially to a free area of the file. Each such change set is called a chunk. All parent pages of the changed B-trees are stored in this chunk as well, so that each chunk also contains the root of each changed map (which is the entry point to read old data). There is no separate index: all data is stored as a list of pages. Per store, the is one additional map that contains the metadata (the list of maps, where the root page of each map is stored, and the list of chunks).
@mvstore_1053_p
@mvstore_1054_p
# There are currently two write operations per chunk: one to store the chunk data (the pages), and one to update the file header (so it points to the latest chunk), but the plan is to write the file header only once in a while, in a way that still allows to open a store very quickly.
@mvstore_1054_p
@mvstore_1055_p
# There is currently no transaction log, no undo log, and there are no in-place updates (however unused chunks are overwritten). To efficiently persist very small transactions, the plan is to support a transaction log where only the deltas is stored, until enough changes have accumulated to persist a chunk. Old versions are kept and are readable until they are no longer needed.
@mvstore_1055_p
@mvstore_1056_p
# The plan is to keep all old data for at least one or two minutes (configurable), so that there are no explicit sync operations required to guarantee data consistency. To reuse disk space, the chunks with the lowest amount of live data are compacted (the live data is simply stored again in the next chunk). To improve data locality and disk space usage, the plan is to automatically defragment and compact data.
@mvstore_1056_p
@mvstore_1057_p
# Compared to regular databases (that use a transaction log, undo log, and main storage area), the log structured storage is simpler, more flexible, and typically needs less disk operations per change, as data is only written once instead of twice or 3 times, and because the B-tree pages are always full (they are stored next to each other) and can be easily compressed. But temporarily, disk space usage might actually be a bit higher than for a regular database, as disk space is not immediately re-used (there are no in-place updates).
@mvstore_1057_h3
@mvstore_1058_h3
#File System Abstraction, File Locking and Online Backup
@mvstore_1058_p
@mvstore_1059_p
# The file system is pluggable (the same file system abstraction is used as H2 uses). Support for encryption is planned using an encrypting file system. Other file system implementations support reading from a compressed zip or tar file.
@mvstore_1059_p
@mvstore_1060_p
# Each store may only be opened once within a JVM. When opening a store, the file is locked in exclusive mode, so that the file can only be changed from within one process. Files can be opened in read-only mode, in which case a shared lock is used.
@mvstore_1060_p
@mvstore_1061_p
# The persisted data can be backed up to a different file at any time, even during write operations (online backup). To do that, automatic disk space reuse needs to be first disabled, so that new data is always appended at the end of the file. Then, the file can be copied (the file handle is available to the application).
@mvstore_1061_h3
@mvstore_1062_h3
#Tools
@mvstore_1062_p
@mvstore_1063_p
# There is a builder for store instances (<code>MVStoreBuilder</code>) with a fluent API to simplify building a store instance.
@mvstore_1063_p
@mvstore_1064_p
# There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
@mvstore_1064_h2
@mvstore_1065_h2
#Similar Projects and Differences to Other Storage Engines
@mvstore_1065_p
@mvstore_1066_p
# Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is written in Java and can easily be embedded in a Java application.
@mvstore_1066_p
@mvstore_1067_p
# The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, and is also a log structured storage, but the H2 license is more liberal.
@mvstore_1067_p
@mvstore_1068_p
# Like SQLite, the MVStore keeps all data in one file. The plan is to make the MVStore easier to use and faster than SQLite on Android (this was not recently tested, however an initial test was successful).
@mvstore_1068_p
@mvstore_1069_p
# The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, and some code is shared between MapDB and JDBM. However, unlike MapDB, the MVStore uses is a log structured storage.
@mvstore_1069_h2
@mvstore_1070_h2
#Current State
@mvstore_1070_p
@mvstore_1071_p
# The code is still very experimental at this stage. The API as well as the behavior will probably change. Features may be added and removed (even thought the main features will stay).
@mvstore_1071_h2
@mvstore_1072_h2
必要条件
@mvstore_1072_p
@mvstore_1073_p
# The MVStore is included in the latest H2 jar file.
@mvstore_1073_p
@mvstore_1074_p
# There are no special requirements to use it. The MVStore should run on any JVM as well as on Android (even thought this was not tested recently).
