Skip to content

H
h2database
提交 871b0a8e authored 作者: Thomas Mueller's avatar Thomas Mueller
浏览文件
操作

A persistent multi-version map: prepare initial release

上级 f2afe926
隐藏空白字符变更
内嵌 并排
正在显示 包含 474 行增加 和 316 行删除
h2/src/docsrc/html/fragments.html
浏览文件 @ 871b0a8e
...@@ -94,6 +94,7 @@ translate --> ...@@ -94,6 +94,7 @@ translate -->
<a href="build.html">Build</a><br /> <a href="build.html">Build</a><br />
<a href="links.html">Links</a><br /> <a href="links.html">Links</a><br />
<a href="jaqu.html">JaQu</a><br /> <a href="jaqu.html">JaQu</a><br />
<a href="mvstore.html">MVStore</a><br />
<br /> <br />
</div> </div>
......
h2/src/docsrc/html/mvstore.html
浏览文件 @ 871b0a8e
...@@ -7,7 +7,7 @@ Initial Developer: H2 Group ...@@ -7,7 +7,7 @@ Initial Developer: H2 Group
--> -->
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en"> <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" /><title> <head><meta http-equiv="Content-Type" content="text/html;charset=utf-8" /><title>
JaQu MVStore
</title><link rel="stylesheet" type="text/css" href="stylesheet.css" /> </title><link rel="stylesheet" type="text/css" href="stylesheet.css" />
<!-- [search] { --> <!-- [search] { -->
<script type="text/javascript" src="navigation.js"></script> <script type="text/javascript" src="navigation.js"></script>
...@@ -34,33 +34,38 @@ JaQu ...@@ -34,33 +34,38 @@ JaQu
Future Plans</a><br /> Future Plans</a><br />
<h2 id="overview">Overview</h2> <h2 id="overview">Overview</h2>
<p>
The MVStore is work in progress, and is planned to be the next storage subsystem of H2.
</p>
<ul><li>MVStore stands for multi-version store. <ul><li>MVStore stands for multi-version store.
</li><li>Each store contains a number of maps (using the <code>java.util.Map</code> interface). </li><li>Each store contains a number of maps (using the <code>java.util.Map</code> interface).
</li><li>The data can be persisted to disk (like a key-value store or a database). </li><li>The data can be persisted to disk (like a key-value store or a database).
</li><li>Fully in-memory operation is supported. </li><li>Fully in-memory operation is supported.
</li><li>It is intended to be fast, simple to use, and small. </li><li>It is intended to be fast, simple to use, and small.
</li><li>Old versions of the data can be read concurrently with all other operations. </li><li>Old versions of the data can be read concurrently with all other operations.
</li><li>Transaction are supported (currently only one transaction can be open at any time). </li><li>Transaction are supported (currently only one transaction at a time).
</li><li>Transactions (even if they are persisted) can be rolled back. </li><li>Transactions (even if they are persisted) can be rolled back.
</li><li>The tool is very modular. It supports pluggable data types / serialization, </li><li>The tool is very modular. It supports pluggable data types / serialization,
pluggable map implementations (B-tree and R-tree currently), BLOB storage, pluggable map implementations (B-tree and R-tree currently), BLOB storage,
and a file system abstraction that supports encryption and compressed and a file system abstraction to support encryption and compressed read-only files.
read-only files.
</li></ul> </li></ul>
<h2 id="example_code">Example Code</h2> <h2 id="example_code">Example Code</h2>
<h3>Map Operations and Versioning</h3>
<p>
The following sample code show how to create a store,
open a map, add some data, and access the current and an old version.
</p>
<pre> <pre>
// open the store (in-memory if fileName is null) // open the store (in-memory if fileName is null)
MVStore s = MVStore.open(fileName); MVStore s = MVStore.open(fileName);
// create/get the map "data" // create/get the map named "data"
// the String.class, String.class will be optional later MVMap<Integer, String> map = s.openMap("data");
MVMap<String, String> map = s.openMap("data",
String.class, String.class);
// add some data // add some data
map.put("1", "Hello"); map.put(1, "Hello");
map.put("2", "World"); map.put(2, "World");
// get the current version, for later use // get the current version, for later use
long oldVersion = s.getCurrentVersion(); long oldVersion = s.getCurrentVersion();
...@@ -71,11 +76,11 @@ s.incrementVersion(); ...@@ -71,11 +76,11 @@ s.incrementVersion();
// more changes, in the new version // more changes, in the new version
// changes can be rolled back if required // changes can be rolled back if required
// changes always go into 'head' (the newest version) // changes always go into 'head' (the newest version)
map.put("1", "Hi"); map.put(1, "Hi");
map.remove("2"); map.remove(2);
// access the old data (before incrementVersion) // access the old data (before incrementVersion)
MVMap<String, String> oldMap = MVMap<Integer, String> oldMap =
map.openVersion(oldVersion); map.openVersion(oldVersion);
// store the newest data to disk // store the newest data to disk
...@@ -84,16 +89,62 @@ s.store(); ...@@ -84,16 +89,62 @@ s.store();
// print the old version (can be done // print the old version (can be done
// concurrently with further modifications) // concurrently with further modifications)
// this will print Hello World // this will print Hello World
System.out.println(oldMap.get("1")); System.out.println(oldMap.get(1));
System.out.println(oldMap.get("2")); System.out.println(oldMap.get(2));
oldMap.close();
// print the newest version ("Hi") // print the newest version ("Hi")
System.out.println(map.get("1")); System.out.println(map.get(1));
// close the store - this doesn't write to disk // close the store - this doesn't write to disk
s.close(); s.close();
</pre> </pre>
<h3>Store Builder</h3>
<p>
The <code>MVStoreBuilder</code> provides a fluid interface
to build a store if more complex configuration options are used.
</p>
<pre>
MVStore s = MVStoreBuilder.
fileBased(fileName).
cacheSizeMB(10).
readOnly().
open();
</pre>
<h3>R-Tree</h3>
<p>
The <code>MVRTreeMap</code> is an R-tree implementation
that supports fast spatial queries.
</p>
<pre>
// create an in-memory store
MVStore s = MVStore.open(null);
// create an R-tree map
// the key has 2 dimensions, the value is a string
MVRTreeMap<String> r = MVRTreeMap.create(2, new ObjectType());
// open the map
r = s.openMap("data", r);
// add two key-value pairs
// the first value is the key id (to make the key unique)
// then the min x, max x, min y, max y
r.add(new SpatialKey(0, -3f, -2f, 2f, 3f), "left");
r.add(new SpatialKey(1, 3f, 4f, 4f, 5f), "right");
// iterate over the intersecting keys
Iterator<SpatialKey> it =
r.findIntersectingKeys(new SpatialKey(0, 0f, 9f, 3f, 6f));
for (SpatialKey k; it.hasNext();) {
k = it.next();
System.out.println(k + ": " + r.get(k));
}
s.close();
</pre>
<h2 id="features">Features</h2> <h2 id="features">Features</h2>
...@@ -103,13 +154,18 @@ Each store supports a set of named maps. ...@@ -103,13 +154,18 @@ Each store supports a set of named maps.
A map is sorted by key, and supports the common lookup operations, A map is sorted by key, and supports the common lookup operations,
including access to the first and last key, iterate over some or all keys, and so on. including access to the first and last key, iterate over some or all keys, and so on.
</p><p> </p><p>
Also supported, and unusual for maps, is fast index lookup, as if the map would be a list Also supported, and very uncommon for maps, is fast index lookup.
(get the key at the given index, get the index of a certain key), and fast skipping within an iterator. The keys of the map can be accessed like a list
Internally, each map is organized in the form of a counted B+-tree. (get the key at the given index, get the index of a certain key).
That means getting the median of two keys is trivial,
and it allows to very quickly count ranges.
The iterator supports fast skipping.
This is possible because internally, each map is organized in the form of a counted B+-tree.
</p><p> </p><p>
In database terms, a map can be a table, where the key of the map is the primary key of the table, In database terms, a map can be used like a table, where the key of the map is the primary key of the table,
and the value is the row. A map can also represent an index, where the key of the map is the key and the value is the row. A map can also represent an index, where the key of the map is the key
of the index, and the value of the map is the primary key of the table. of the index, and the value of the map is the primary key of the table (for non-unique indexes
the key of the map must also contain the primary key).
</p> </p>
<h3>Versions / Transactions</h3> <h3>Versions / Transactions</h3>
...@@ -189,9 +245,11 @@ use less memory on average. ...@@ -189,9 +245,11 @@ use less memory on average.
<h3>Pluggable Data Types</h3> <h3>Pluggable Data Types</h3>
<p> <p>
Serialization is pluggable. The default serialization currently only supports string values and keys, Serialization is pluggable. The default serialization currently supports many common data types,
but there is a sample plugin that supports many common data types, and uses Java serialization for other objects. The following classes are currently directly supported:
and uses Java serialization for other objects. <code>Boolean, Byte, Short, Character, Integer, Long, Float, Double, BigInteger, BigDecimal,
byte[], char[], int[], long[], String, UUID</code>.
The plan is to add more common classes (date, time, timestamp, object array).
</p><p> </p><p>
Parameterized data types are supported Parameterized data types are supported
(for example one could build a string data type that limits the length for some reason). (for example one could build a string data type that limits the length for some reason).
...@@ -232,7 +290,7 @@ which should be resistant against scan operations. ...@@ -232,7 +290,7 @@ which should be resistant against scan operations.
Concurrent modification operations on the maps are currently not supported, Concurrent modification operations on the maps are currently not supported,
however it is planned to support an additional map implementation however it is planned to support an additional map implementation
that supports concurrent writes that supports concurrent writes
(at the cost of speed if used in a single thread, same as ConcurrentHashMap). (at the cost of speed if used in a single thread, same as <code>ConcurrentHashMap</code>).
</p> </p>
<h3>File System Abstraction, File Locking and Online Backup</h3> <h3>File System Abstraction, File Locking and Online Backup</h3>
...@@ -251,9 +309,18 @@ Files can be opened in read-only mode, in which case a shared lock is used. ...@@ -251,9 +309,18 @@ Files can be opened in read-only mode, in which case a shared lock is used.
The persisted data can be backed up to a different file at any time, The persisted data can be backed up to a different file at any time,
even during write operations (online backup). even during write operations (online backup).
To do that, automatic disk space reuse needs to be first disabled, so that To do that, automatic disk space reuse needs to be first disabled, so that
new data is always appended at the end of the file. Then, the file new data is always appended at the end of the file.
Then, the file can be copied (the file handle is available to the application).
</p> </p>
<h3>Tools</h3>
<p>
There is a builder for store instances (<code>MVStoreBuilder</code>)
with a fluent API to simplify building a store instance.
</p>
<p>
There is a tool (<code>MVStoreTool</code>) to dump the contents of a file.
</p>
<h2 id="differences">Similar Projects and Differences to Other Storage Engines</h2> <h2 id="differences">Similar Projects and Differences to Other Storage Engines</h2>
<p> <p>
...@@ -261,7 +328,7 @@ Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is wr ...@@ -261,7 +328,7 @@ Unlike similar storage engines like LevelDB and Kyoto Cabinet, the MVStore is wr
and can easily be embedded in a Java application. and can easily be embedded in a Java application.
</p><p> </p><p>
The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java, The MVStore is somewhat similar to the Berkeley DB Java Edition because it is also written in Java,
and also a log structured storage, but the MVStore licensing is more liberal. and is also a log structured storage, but the H2 license is more liberal.
</p><p> </p><p>
Like SQLite, the MVStore keeps all data in one file. Like SQLite, the MVStore keeps all data in one file.
The plan is to make the MVStore easier to use and faster than SQLite on Android The plan is to make the MVStore easier to use and faster than SQLite on Android
...@@ -269,7 +336,7 @@ The plan is to make the MVStore easier to use and faster than SQLite on Android ...@@ -269,7 +336,7 @@ The plan is to make the MVStore easier to use and faster than SQLite on Android
</p><p> </p><p>
The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek, The API of the MVStore is similar to MapDB (previously known as JDBM) from Jan Kotek,
and some code is shared between MapDB and JDBM. and some code is shared between MapDB and JDBM.
However, the MVStore is a log structured storage. However, unlike MapDB, the MVStore uses is a log structured storage.
</p> </p>
<h2 id="current_state">Current State</h2> <h2 id="current_state">Current State</h2>
......
h2/src/test/org/h2/test/TestAll.java
浏览文件 @ 871b0a8e
...@@ -555,120 +555,120 @@ kill -9 `jps -l | grep "org.h2.test." | cut -d " " -f 1` ...@@ -555,120 +555,120 @@ kill -9 `jps -l | grep "org.h2.test." | cut -d " " -f 1`
beforeTest(); beforeTest();
// db // db
new TestScriptSimple().runTest(this); // new TestScriptSimple().runTest(this);
new TestScript().runTest(this); // new TestScript().runTest(this);
new TestAlter().runTest(this); // new TestAlter().runTest(this);
new TestAlterSchemaRename().runTest(this); // new TestAlterSchemaRename().runTest(this);
new TestAutoRecompile().runTest(this); // new TestAutoRecompile().runTest(this);
new TestBitField().runTest(this); // new TestBitField().runTest(this);
new TestBackup().runTest(this); // new TestBackup().runTest(this);
new TestBigDb().runTest(this); // new TestBigDb().runTest(this);
new TestBigResult().runTest(this); // new TestBigResult().runTest(this);
new TestCases().runTest(this); // new TestCases().runTest(this);
new TestCheckpoint().runTest(this); // new TestCheckpoint().runTest(this);
new TestCluster().runTest(this); // new TestCluster().runTest(this);
new TestCompatibility().runTest(this); // new TestCompatibility().runTest(this);
new TestCsv().runTest(this); // new TestCsv().runTest(this);
new TestDateStorage().runTest(this); // new TestDateStorage().runTest(this);
new TestDeadlock().runTest(this); // new TestDeadlock().runTest(this);
new TestEncryptedDb().runTest(this); // new TestEncryptedDb().runTest(this);
new TestExclusive().runTest(this); // new TestExclusive().runTest(this);
new TestFullText().runTest(this); // new TestFullText().runTest(this);
new TestFunctionOverload().runTest(this); // new TestFunctionOverload().runTest(this);
new TestFunctions().runTest(this); // new TestFunctions().runTest(this);
new TestInit().runTest(this); // new TestInit().runTest(this);
new TestIndex().runTest(this); // new TestIndex().runTest(this);
new TestLargeBlob().runTest(this); // new TestLargeBlob().runTest(this);
new TestLinkedTable().runTest(this); // new TestLinkedTable().runTest(this);
new TestListener().runTest(this); // new TestListener().runTest(this);
new TestLob().runTest(this); // new TestLob().runTest(this);
new TestMemoryUsage().runTest(this); // new TestMemoryUsage().runTest(this);
new TestMultiConn().runTest(this); // new TestMultiConn().runTest(this);
new TestMultiDimension().runTest(this); // new TestMultiDimension().runTest(this);
new TestMultiThread().runTest(this); // new TestMultiThread().runTest(this);
new TestMultiThreadedKernel().runTest(this); // new TestMultiThreadedKernel().runTest(this);
new TestOpenClose().runTest(this); // new TestOpenClose().runTest(this);
new TestOptimizations().runTest(this); // new TestOptimizations().runTest(this);
new TestOutOfMemory().runTest(this); // new TestOutOfMemory().runTest(this);
new TestPowerOff().runTest(this); // new TestPowerOff().runTest(this);
new TestQueryCache().runTest(this); // new TestQueryCache().runTest(this);
new TestReadOnly().runTest(this); // new TestReadOnly().runTest(this);
new TestRecursiveQueries().runTest(this); // new TestRecursiveQueries().runTest(this);
new TestRights().runTest(this); // new TestRights().runTest(this);
new TestRunscript().runTest(this); // new TestRunscript().runTest(this);
new TestSQLInjection().runTest(this); // new TestSQLInjection().runTest(this);
new TestSessionsLocks().runTest(this); // new TestSessionsLocks().runTest(this);
new TestSelectCountNonNullColumn().runTest(this); // new TestSelectCountNonNullColumn().runTest(this);
new TestSequence().runTest(this); // new TestSequence().runTest(this);
new TestSpaceReuse().runTest(this); // new TestSpaceReuse().runTest(this);
new TestSpeed().runTest(this); // new TestSpeed().runTest(this);
new TestTableEngines().runTest(this); // new TestTableEngines().runTest(this);
new TestTempTables().runTest(this); // new TestTempTables().runTest(this);
new TestTransaction().runTest(this); // new TestTransaction().runTest(this);
new TestTriggersConstraints().runTest(this); // new TestTriggersConstraints().runTest(this);
new TestTwoPhaseCommit().runTest(this); // new TestTwoPhaseCommit().runTest(this);
new TestView().runTest(this); // new TestView().runTest(this);
new TestViewAlterTable().runTest(this); // new TestViewAlterTable().runTest(this);
new TestViewDropView().runTest(this); // new TestViewDropView().runTest(this);
//
// jaqu // // jaqu
new AliasMapTest().runTest(this); // new AliasMapTest().runTest(this);
new AnnotationsTest().runTest(this); // new AnnotationsTest().runTest(this);
new ClobTest().runTest(this); // new ClobTest().runTest(this);
new ModelsTest().runTest(this); // new ModelsTest().runTest(this);
new SamplesTest().runTest(this); // new SamplesTest().runTest(this);
new UpdateTest().runTest(this); // new UpdateTest().runTest(this);
//
// jdbc // // jdbc
new TestBatchUpdates().runTest(this); // new TestBatchUpdates().runTest(this);
new TestCallableStatement().runTest(this); // new TestCallableStatement().runTest(this);
new TestCancel().runTest(this); // new TestCancel().runTest(this);
new TestDatabaseEventListener().runTest(this); // new TestDatabaseEventListener().runTest(this);
new TestDriver().runTest(this); // new TestDriver().runTest(this);
new TestJavaObject().runTest(this); // new TestJavaObject().runTest(this);
new TestLimitUpdates().runTest(this); // new TestLimitUpdates().runTest(this);
new TestLobApi().runTest(this); // new TestLobApi().runTest(this);
new TestManyJdbcObjects().runTest(this); // new TestManyJdbcObjects().runTest(this);
new TestMetaData().runTest(this); // new TestMetaData().runTest(this);
new TestNativeSQL().runTest(this); // new TestNativeSQL().runTest(this);
new TestPreparedStatement().runTest(this); // new TestPreparedStatement().runTest(this);
new TestResultSet().runTest(this); // new TestResultSet().runTest(this);
new TestStatement().runTest(this); // new TestStatement().runTest(this);
new TestTransactionIsolation().runTest(this); // new TestTransactionIsolation().runTest(this);
new TestUpdatableResultSet().runTest(this); // new TestUpdatableResultSet().runTest(this);
new TestZloty().runTest(this); // new TestZloty().runTest(this);
//
// jdbcx // // jdbcx
new TestConnectionPool().runTest(this); // new TestConnectionPool().runTest(this);
new TestDataSource().runTest(this); // new TestDataSource().runTest(this);
new TestXA().runTest(this); // new TestXA().runTest(this);
new TestXASimple().runTest(this); // new TestXASimple().runTest(this);
//
// server // // server
new TestAutoServer().runTest(this); // new TestAutoServer().runTest(this);
new TestNestedLoop().runTest(this); // new TestNestedLoop().runTest(this);
new TestWeb().runTest(this); // new TestWeb().runTest(this);
//
// mvcc & row level locking // // mvcc & row level locking
new TestMvcc1().runTest(this); // new TestMvcc1().runTest(this);
new TestMvcc2().runTest(this); // new TestMvcc2().runTest(this);
new TestMvcc3().runTest(this); // new TestMvcc3().runTest(this);
new TestMvccMultiThreaded().runTest(this); // new TestMvccMultiThreaded().runTest(this);
new TestRowLocks().runTest(this); // new TestRowLocks().runTest(this);
//
// synth // // synth
new TestBtreeIndex().runTest(this); // new TestBtreeIndex().runTest(this);
new TestDiskFull().runTest(this); // new TestDiskFull().runTest(this);
new TestCrashAPI().runTest(this); // new TestCrashAPI().runTest(this);
new TestFuzzOptimizations().runTest(this); // new TestFuzzOptimizations().runTest(this);
new TestLimit().runTest(this); // new TestLimit().runTest(this);
new TestRandomSQL().runTest(this); // new TestRandomSQL().runTest(this);
new TestRandomCompare().runTest(this); // new TestRandomCompare().runTest(this);
new TestKillRestart().runTest(this); // new TestKillRestart().runTest(this);
new TestKillRestartMulti().runTest(this); // new TestKillRestartMulti().runTest(this);
new TestMultiThreaded().runTest(this); // new TestMultiThreaded().runTest(this);
new TestOuterJoins().runTest(this); // new TestOuterJoins().runTest(this);
new TestNestedJoins().runTest(this); // new TestNestedJoins().runTest(this);
afterTest(); afterTest();
} }
......
h2/src/test/org/h2/test/store/IntegerType.java deleted 100644 → 0
浏览文件 @ f2afe926
/*
* Copyright 2004-2011 H2 Group. Multiple-Licensed under the H2 License,
* Version 1.0, and under the Eclipse Public License, Version 1.0
* (http://h2database.com/html/license.html).
* Initial Developer: H2 Group
*/
package org.h2.test.store;
import java.nio.ByteBuffer;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.DataUtils;
/**
* An integer type.
*/
class IntegerType implements DataType {
public static final IntegerType INSTANCE = new IntegerType();
public int compare(Object a, Object b) {
return ((Integer) a).compareTo((Integer) b);
}
public int getMaxLength(Object obj) {
return DataUtils.MAX_VAR_INT_LEN;
}
public int getMemory(Object obj) {
return 20;
}
public Integer read(ByteBuffer buff) {
return DataUtils.readVarInt(buff);
}
public void write(ByteBuffer buff, Object x) {
DataUtils.writeVarInt(buff, (Integer) x);
}
public String asString() {
return "i";
}
}
h2/src/test/org/h2/test/store/RowType.java
浏览文件 @ 871b0a8e
...@@ -7,9 +7,9 @@ ...@@ -7,9 +7,9 @@
package org.h2.test.store; package org.h2.test.store;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.DataUtils; import org.h2.dev.store.btree.DataUtils;
import org.h2.dev.store.btree.DataTypeFactory; import org.h2.dev.store.type.DataType;
import org.h2.dev.store.type.DataTypeFactory;
import org.h2.util.StringUtils; import org.h2.util.StringUtils;
/** /**
...@@ -17,6 +17,8 @@ import org.h2.util.StringUtils; ...@@ -17,6 +17,8 @@ import org.h2.util.StringUtils;
*/ */
public class RowType implements DataType { public class RowType implements DataType {
static final String PREFIX = "org.h2.test.store.row";
private final DataType[] types; private final DataType[] types;
RowType(DataType[] types) { RowType(DataType[] types) {
...@@ -86,7 +88,7 @@ public class RowType implements DataType { ...@@ -86,7 +88,7 @@ public class RowType implements DataType {
public String asString() { public String asString() {
StringBuilder buff = new StringBuilder(); StringBuilder buff = new StringBuilder();
buff.append('r'); buff.append(PREFIX);
buff.append('('); buff.append('(');
for (int i = 0; i < types.length; i++) { for (int i = 0; i < types.length; i++) {
if (i > 0) { if (i > 0) {
...@@ -106,10 +108,10 @@ public class RowType implements DataType { ...@@ -106,10 +108,10 @@ public class RowType implements DataType {
* @return the row type * @return the row type
*/ */
static RowType fromString(String t, DataTypeFactory factory) { static RowType fromString(String t, DataTypeFactory factory) {
if (!t.startsWith("r(") || !t.endsWith(")")) { if (!t.startsWith(PREFIX) || !t.endsWith(")")) {
throw new RuntimeException("Unknown type: " + t); throw new RuntimeException("Unknown type: " + t);
} }
t = t.substring(2, t.length() - 1); t = t.substring(PREFIX.length(), t.length() - 1);
String[] array = StringUtils.arraySplit(t, ',', false); String[] array = StringUtils.arraySplit(t, ',', false);
DataType[] types = new DataType[array.length]; DataType[] types = new DataType[array.length];
for (int i = 0; i < array.length; i++) { for (int i = 0; i < array.length; i++) {
......
h2/src/test/org/h2/test/store/SampleTypeFactory.java 0 → 100644
浏览文件 @ 871b0a8e
/*
* Copyright 2004-2011 H2 Group. Multiple-Licensed under the H2 License, Version
* 1.0, and under the Eclipse Public License, Version 1.0
* (http://h2database.com/html/license.html). Initial Developer: H2 Group
*/
package org.h2.test.store;
import org.h2.dev.store.type.DataType;
import org.h2.dev.store.type.DataTypeFactory;
/**
* A data type factory.
*/
public class SampleTypeFactory implements DataTypeFactory {
private DataTypeFactory parent;
@Override
public void setParent(DataTypeFactory parent) {
this.parent = parent;
}
@Override
public DataType buildDataType(String s) {
// if ("org.h2.test.store.int".equals(s)) {
// return new IntegerType();
// } else
if (s.startsWith(RowType.PREFIX)) {
return RowType.fromString(s, this);
}
return parent.buildDataType(s);
}
@Override
public String getDataType(Class<?> objectClass) {
// if (objectClass == Integer.class) {
// return "org.h2.test.store.int";
// }
return parent.getDataType(objectClass);
}
}
h2/src/test/org/h2/test/store/SequenceMap.java
浏览文件 @ 871b0a8e
...@@ -16,7 +16,7 @@ import org.h2.dev.store.btree.MVStore; ...@@ -16,7 +16,7 @@ import org.h2.dev.store.btree.MVStore;
/** /**
* A custom map returning the keys and values values 1 .. 10. * A custom map returning the keys and values values 1 .. 10.
*/ */
public class SequenceMap extends MVMap<Integer, String> { public class SequenceMap extends MVMap<Long, Long> {
/** /**
* The minimum value. * The minimum value.
...@@ -29,7 +29,7 @@ public class SequenceMap extends MVMap<Integer, String> { ...@@ -29,7 +29,7 @@ public class SequenceMap extends MVMap<Integer, String> {
int max = 10; int max = 10;
public SequenceMap() { public SequenceMap() {
super(IntegerType.INSTANCE, IntegerType.INSTANCE); super(null, null);
} }
public void open(MVStore store, HashMap<String, String> config) { public void open(MVStore store, HashMap<String, String> config) {
...@@ -37,14 +37,14 @@ public class SequenceMap extends MVMap<Integer, String> { ...@@ -37,14 +37,14 @@ public class SequenceMap extends MVMap<Integer, String> {
setReadOnly(true); setReadOnly(true);
} }
public Set<Integer> keySet() { public Set<Long> keySet() {
return new AbstractSet<Integer>() { return new AbstractSet<Long>() {
@Override @Override
public Iterator<Integer> iterator() { public Iterator<Long> iterator() {
return new Iterator<Integer>() { return new Iterator<Long>() {
int x = min; long x = min;
@Override @Override
public boolean hasNext() { public boolean hasNext() {
...@@ -52,8 +52,8 @@ public class SequenceMap extends MVMap<Integer, String> { ...@@ -52,8 +52,8 @@ public class SequenceMap extends MVMap<Integer, String> {
} }
@Override @Override
public Integer next() { public Long next() {
return Integer.valueOf(x++); return Long.valueOf(x++);
} }
@Override @Override
......
h2/src/test/org/h2/test/store/TestConcurrent.java
浏览文件 @ 871b0a8e
...@@ -17,6 +17,7 @@ import java.util.Random; ...@@ -17,6 +17,7 @@ import java.util.Random;
import org.h2.dev.store.btree.MVMap; import org.h2.dev.store.btree.MVMap;
import org.h2.dev.store.btree.MVStore; import org.h2.dev.store.btree.MVStore;
import org.h2.dev.store.btree.MVStoreBuilder; import org.h2.dev.store.btree.MVStoreBuilder;
import org.h2.dev.store.type.ObjectTypeFactory;
import org.h2.store.fs.FileUtils; import org.h2.store.fs.FileUtils;
import org.h2.test.TestBase; import org.h2.test.TestBase;
import org.h2.util.Task; import org.h2.util.Task;
...@@ -132,8 +133,8 @@ public class TestConcurrent extends TestMVStore { ...@@ -132,8 +133,8 @@ public class TestConcurrent extends TestMVStore {
private void testConcurrentIterate() { private void testConcurrentIterate() {
MVStore s = MVStoreBuilder.inMemory(). MVStore s = MVStoreBuilder.inMemory().
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
s.setMaxPageSize(3); s.setPageSize(3);
final MVMap<Integer, Integer> map = s.openMap("test"); final MVMap<Integer, Integer> map = s.openMap("test");
final int len = 10; final int len = 10;
final Random r = new Random(); final Random r = new Random();
......
h2/src/test/org/h2/test/store/TestMVRTree.java
浏览文件 @ 871b0a8e
...@@ -21,7 +21,10 @@ import javax.imageio.ImageIO; ...@@ -21,7 +21,10 @@ import javax.imageio.ImageIO;
import javax.imageio.ImageWriter; import javax.imageio.ImageWriter;
import javax.imageio.stream.FileImageOutputStream; import javax.imageio.stream.FileImageOutputStream;
import org.h2.dev.store.btree.MVStore; import org.h2.dev.store.btree.MVStore;
import org.h2.dev.store.btree.StringType; import org.h2.dev.store.rtree.MVRTreeMap;
import org.h2.dev.store.rtree.SpatialKey;
import org.h2.dev.store.type.ObjectType;
import org.h2.dev.store.type.StringType;
import org.h2.store.fs.FileUtils; import org.h2.store.fs.FileUtils;
import org.h2.test.TestBase; import org.h2.test.TestBase;
import org.h2.util.New; import org.h2.util.New;
...@@ -41,10 +44,38 @@ public class TestMVRTree extends TestMVStore { ...@@ -41,10 +44,38 @@ public class TestMVRTree extends TestMVStore {
} }
public void test() { public void test() {
testExample();
testMany(); testMany();
testSimple(); testSimple();
testRandom(); testRandom();
testCustomMapType(); }
private void testExample() {
// create an in-memory store
MVStore s = MVStore.open(null);
// create an R-tree map
// the key has 2 dimensions, the value is a string
MVRTreeMap<String> r = MVRTreeMap.create(2, new ObjectType());
// open the map
r = s.openMap("data", r);
// add two key-value pairs
// the first value is the key id (to make the key unique)
// then the min x, max x, min y, max y
r.add(new SpatialKey(0, -3f, -2f, 2f, 3f), "left");
r.add(new SpatialKey(1, 3f, 4f, 4f, 5f), "right");
// iterate over the intersecting keys
Iterator<SpatialKey> it = r.findIntersectingKeys(
new SpatialKey(0, 0f, 9f, 3f, 6f));
for (SpatialKey k; it.hasNext();) {
k = it.next();
// System.out.println(k + ": " + r.get(k));
assertTrue(k != null);
}
s.close();
} }
private void testMany() { private void testMany() {
...@@ -276,19 +307,4 @@ public class TestMVRTree extends TestMVStore { ...@@ -276,19 +307,4 @@ public class TestMVRTree extends TestMVStore {
s.close(); s.close();
} }
private void testCustomMapType() {
String fileName = getBaseDir() + "/testMapType.h3";
FileUtils.delete(fileName);
MVStore s;
s = openStore(fileName);
SequenceMap seq = new SequenceMap();
seq = s.openMap("data", seq);
StringBuilder buff = new StringBuilder();
for (int x : seq.keySet()) {
buff.append(x).append(';');
}
assertEquals("1;2;3;4;5;6;7;8;9;10;", buff.toString());
s.close();
}
} }
h2/src/test/org/h2/test/store/TestMVStore.java
浏览文件 @ 871b0a8e
...@@ -14,11 +14,13 @@ import java.util.Map; ...@@ -14,11 +14,13 @@ import java.util.Map;
import java.util.Random; import java.util.Random;
import java.util.TreeMap; import java.util.TreeMap;
import org.h2.dev.store.btree.Cursor; import org.h2.dev.store.btree.Cursor;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.MVMap; import org.h2.dev.store.btree.MVMap;
import org.h2.dev.store.btree.MVStore; import org.h2.dev.store.btree.MVStore;
import org.h2.dev.store.btree.MVStoreBuilder; import org.h2.dev.store.btree.MVStoreBuilder;
import org.h2.dev.store.btree.StringType; import org.h2.dev.store.type.DataType;
import org.h2.dev.store.type.ObjectType;
import org.h2.dev.store.type.ObjectTypeFactory;
import org.h2.dev.store.type.StringType;
import org.h2.store.fs.FilePath; import org.h2.store.fs.FilePath;
import org.h2.store.fs.FileUtils; import org.h2.store.fs.FileUtils;
import org.h2.test.TestBase; import org.h2.test.TestBase;
...@@ -40,6 +42,7 @@ public class TestMVStore extends TestBase { ...@@ -40,6 +42,7 @@ public class TestMVStore extends TestBase {
public void test() throws Exception { public void test() throws Exception {
FileUtils.deleteRecursive(getBaseDir(), true); FileUtils.deleteRecursive(getBaseDir(), true);
testCustomMapType();
testCacheSize(); testCacheSize();
testConcurrentOpen(); testConcurrentOpen();
testFileHeader(); testFileHeader();
...@@ -70,12 +73,27 @@ public class TestMVStore extends TestBase { ...@@ -70,12 +73,27 @@ public class TestMVStore extends TestBase {
testSimple(); testSimple();
} }
private void testCustomMapType() {
String fileName = getBaseDir() + "/testMapType.h3";
FileUtils.delete(fileName);
MVStore s;
s = openStore(fileName);
SequenceMap seq = new SequenceMap();
seq = s.openMap("data", seq);
StringBuilder buff = new StringBuilder();
for (long x : seq.keySet()) {
buff.append(x).append(';');
}
assertEquals("1;2;3;4;5;6;7;8;9;10;", buff.toString());
s.close();
}
private void testCacheSize() { private void testCacheSize() {
String fileName = getBaseDir() + "/testCacheSize.h3"; String fileName = getBaseDir() + "/testCacheSize.h3";
MVStore s; MVStore s;
MVMap<Integer, String> map; MVMap<Integer, String> map;
s = MVStoreBuilder.fileBased(fileName). s = MVStoreBuilder.fileBased(fileName).
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
map = s.openMap("test"); map = s.openMap("test");
// add 10 MB of data // add 10 MB of data
for (int i = 0; i < 1024; i++) { for (int i = 0; i < 1024; i++) {
...@@ -89,7 +107,7 @@ public class TestMVStore extends TestBase { ...@@ -89,7 +107,7 @@ public class TestMVStore extends TestBase {
for (int cacheSize = 0; cacheSize <= 6; cacheSize += 4) { for (int cacheSize = 0; cacheSize <= 6; cacheSize += 4) {
s = MVStoreBuilder.fileBased(fileName). s = MVStoreBuilder.fileBased(fileName).
cacheSizeMB(1 + 3 * cacheSize). cacheSizeMB(1 + 3 * cacheSize).
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
map = s.openMap("test"); map = s.openMap("test");
for (int i = 0; i < 1024; i += 128) { for (int i = 0; i < 1024; i += 128) {
for (int j = 0; j < i; j++) { for (int j = 0; j < i; j++) {
...@@ -179,7 +197,7 @@ public class TestMVStore extends TestBase { ...@@ -179,7 +197,7 @@ public class TestMVStore extends TestBase {
private void testIndexSkip() { private void testIndexSkip() {
MVStore s = openStore(null); MVStore s = openStore(null);
s.setMaxPageSize(4); s.setPageSize(4);
MVMap<Integer, Integer> map = s.openMap("test"); MVMap<Integer, Integer> map = s.openMap("test");
for (int i = 0; i < 100; i += 2) { for (int i = 0; i < 100; i += 2) {
map.put(i, 10 * i); map.put(i, 10 * i);
...@@ -240,7 +258,7 @@ public class TestMVStore extends TestBase { ...@@ -240,7 +258,7 @@ public class TestMVStore extends TestBase {
for (int i = 3; i < 20; i++) { for (int i = 3; i < 20; i++) {
s = openStore(null); s = openStore(null);
s.setMaxPageSize(4); s.setPageSize(4);
map = s.openMap("test"); map = s.openMap("test");
for (int j = 3; j < i; j++) { for (int j = 3; j < i; j++) {
map.put(j * 2, j * 20); map.put(j * 2, j * 20);
...@@ -310,7 +328,7 @@ public class TestMVStore extends TestBase { ...@@ -310,7 +328,7 @@ public class TestMVStore extends TestBase {
MVStore s; MVStore s;
Map<Integer, Integer> map; Map<Integer, Integer> map;
s = MVStoreBuilder.inMemory(). s = MVStoreBuilder.inMemory().
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
map = s.openMap("test"); map = s.openMap("test");
int len = 100; int len = 100;
for (int i = 0; i < len; i++) { for (int i = 0; i < len; i++) {
...@@ -336,7 +354,7 @@ public class TestMVStore extends TestBase { ...@@ -336,7 +354,7 @@ public class TestMVStore extends TestBase {
MVStore s; MVStore s;
Map<Object, Object> map; Map<Object, Object> map;
s = MVStoreBuilder.fileBased(fileName). s = MVStoreBuilder.fileBased(fileName).
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
map = s.openMap("test"); map = s.openMap("test");
map.put(1, "Hello"); map.put(1, "Hello");
map.put("2", 200); map.put("2", 200);
...@@ -344,7 +362,7 @@ public class TestMVStore extends TestBase { ...@@ -344,7 +362,7 @@ public class TestMVStore extends TestBase {
s.store(); s.store();
s.close(); s.close();
s = MVStoreBuilder.fileBased(fileName). s = MVStoreBuilder.fileBased(fileName).
with(new SampleDataTypeFactory()).open(); with(new ObjectTypeFactory()).open();
map = s.openMap("test"); map = s.openMap("test");
assertEquals("Hello", map.get(1).toString()); assertEquals("Hello", map.get(1).toString());
assertEquals(200, ((Integer) map.get("2")).intValue()); assertEquals(200, ((Integer) map.get("2")).intValue());
...@@ -362,14 +380,12 @@ public class TestMVStore extends TestBase { ...@@ -362,14 +380,12 @@ public class TestMVStore extends TestBase {
// open the store (in-memory if fileName is null) // open the store (in-memory if fileName is null)
MVStore s = MVStore.open(fileName); MVStore s = MVStore.open(fileName);
// create/get the map "data" // create/get the map named "data"
// the String.class, String.class will be optional later MVMap<Integer, String> map = s.openMap("data");
MVMap<String, String> map = s.openMap("data",
String.class, String.class);
// add some data // add some data
map.put("1", "Hello"); map.put(1, "Hello");
map.put("2", "World"); map.put(2, "World");
// get the current version, for later use // get the current version, for later use
long oldVersion = s.getCurrentVersion(); long oldVersion = s.getCurrentVersion();
...@@ -380,11 +396,11 @@ public class TestMVStore extends TestBase { ...@@ -380,11 +396,11 @@ public class TestMVStore extends TestBase {
// more changes, in the new version // more changes, in the new version
// changes can be rolled back if required // changes can be rolled back if required
// changes always go into 'head' (the newest version) // changes always go into 'head' (the newest version)
map.put("1", "Hi"); map.put(1, "Hi");
map.remove("2"); map.remove(2);
// access the old data (before incrementVersion) // access the old data (before incrementVersion)
MVMap<String, String> oldMap = MVMap<Integer, String> oldMap =
map.openVersion(oldVersion); map.openVersion(oldVersion);
// store the newest data to disk // store the newest data to disk
...@@ -393,12 +409,12 @@ public class TestMVStore extends TestBase { ...@@ -393,12 +409,12 @@ public class TestMVStore extends TestBase {
// print the old version (can be done // print the old version (can be done
// concurrently with further modifications) // concurrently with further modifications)
// this will print Hello World // this will print Hello World
// System.out.println(oldMap.get("1")); // System.out.println(oldMap.get(1));
// System.out.println(oldMap.get("2")); // System.out.println(oldMap.get(2));
oldMap.close(); oldMap.close();
// print the newest version ("Hi") // print the newest version ("Hi")
// System.out.println(map.get("1")); // System.out.println(map.get(1));
// close the store - this doesn't write to disk // close the store - this doesn't write to disk
s.close(); s.close();
...@@ -429,7 +445,7 @@ public class TestMVStore extends TestBase { ...@@ -429,7 +445,7 @@ public class TestMVStore extends TestBase {
String fileName = getBaseDir() + "/testIterate.h3"; String fileName = getBaseDir() + "/testIterate.h3";
FileUtils.delete(fileName); FileUtils.delete(fileName);
MVStore s = openStore(fileName); MVStore s = openStore(fileName);
s.setMaxPageSize(6); s.setPageSize(6);
MVMap<Integer, String> m = s.openMap("data", Integer.class, String.class); MVMap<Integer, String> m = s.openMap("data", Integer.class, String.class);
for (int i = 0; i < 100; i++) { for (int i = 0; i < 100; i++) {
m.put(i, "Hi"); m.put(i, "Hi");
...@@ -538,14 +554,14 @@ public class TestMVStore extends TestBase { ...@@ -538,14 +554,14 @@ public class TestMVStore extends TestBase {
MVStore s; MVStore s;
MVMap<Integer, String> m; MVMap<Integer, String> m;
s = openStore(fileName); s = openStore(fileName);
s.setMaxPageSize(700); s.setPageSize(700);
m = s.openMap("data", Integer.class, String.class); m = s.openMap("data", Integer.class, String.class);
for (int i = 0; i < 1000; i++) { for (int i = 0; i < 1000; i++) {
m.put(i, "Hello World"); m.put(i, "Hello World");
assertEquals(i + 1, m.size()); assertEquals(i + 1, m.size());
} }
assertEquals(1000, m.size()); assertEquals(1000, m.size());
assertEquals(279, s.getUnsavedPageCount()); assertEquals(284, s.getUnsavedPageCount());
s.store(); s.store();
assertEquals(3, s.getFileWriteCount()); assertEquals(3, s.getFileWriteCount());
s.close(); s.close();
...@@ -556,7 +572,7 @@ public class TestMVStore extends TestBase { ...@@ -556,7 +572,7 @@ public class TestMVStore extends TestBase {
assertEquals(0, m.size()); assertEquals(0, m.size());
s.store(); s.store();
// ensure only nodes are read, but not leaves // ensure only nodes are read, but not leaves
assertEquals(36, s.getFileReadCount()); assertEquals(41, s.getFileReadCount());
assertEquals(2, s.getFileWriteCount()); assertEquals(2, s.getFileWriteCount());
s.close(); s.close();
} }
...@@ -652,7 +668,7 @@ public class TestMVStore extends TestBase { ...@@ -652,7 +668,7 @@ public class TestMVStore extends TestBase {
FileUtils.delete(fileName); FileUtils.delete(fileName);
MVStore s = openStore(fileName); MVStore s = openStore(fileName);
assertEquals(0, s.getCurrentVersion()); assertEquals(0, s.getCurrentVersion());
s.setMaxPageSize(5); s.setPageSize(5);
MVMap<String, String> m = s.openMap("data", String.class, String.class); MVMap<String, String> m = s.openMap("data", String.class, String.class);
s.rollbackTo(0); s.rollbackTo(0);
assertTrue(m.isClosed()); assertTrue(m.isClosed());
...@@ -753,11 +769,11 @@ public class TestMVStore extends TestBase { ...@@ -753,11 +769,11 @@ public class TestMVStore extends TestBase {
FileUtils.delete(fileName); FileUtils.delete(fileName);
MVStore s = openStore(fileName); MVStore s = openStore(fileName);
// s.setCompressor(null); // s.setCompressor(null);
s.setMaxPageSize(40); s.setPageSize(40);
MVMap<Integer, Object[]> m = new MVMap<Integer, Object[]>( MVMap<Integer, Object[]> m = new MVMap<Integer, Object[]>(
IntegerType.INSTANCE, new ObjectType(),
new RowType(new DataType[] { new RowType(new DataType[] {
IntegerType.INSTANCE, new ObjectType(),
StringType.INSTANCE, StringType.INSTANCE,
StringType.INSTANCE StringType.INSTANCE
}) })
...@@ -965,26 +981,9 @@ public class TestMVStore extends TestBase { ...@@ -965,26 +981,9 @@ public class TestMVStore extends TestBase {
} }
private void testKeyValueClasses() { private void testKeyValueClasses() {
MVStore s;
s = MVStore.open(null);
s.openMap("test", String.class, String.class);
try {
s.openMap("unsupportedKey", ArrayList.class, String.class);
fail();
} catch (RuntimeException e) {
// expected
}
try {
s.openMap("unsupportedValue", String.class, ArrayList.class);
fail();
} catch (RuntimeException e) {
// expected
}
s.close();
String fileName = getBaseDir() + "/testKeyValueClasses.h3"; String fileName = getBaseDir() + "/testKeyValueClasses.h3";
FileUtils.delete(fileName); FileUtils.delete(fileName);
s = openStore(fileName); MVStore s = openStore(fileName);
MVMap<Integer, String> is = s.openMap("intString", Integer.class, String.class); MVMap<Integer, String> is = s.openMap("intString", Integer.class, String.class);
is.put(1, "Hello"); is.put(1, "Hello");
MVMap<Integer, Integer> ii = s.openMap("intInt", Integer.class, Integer.class); MVMap<Integer, Integer> ii = s.openMap("intInt", Integer.class, Integer.class);
...@@ -1089,8 +1088,8 @@ public class TestMVStore extends TestBase { ...@@ -1089,8 +1088,8 @@ public class TestMVStore extends TestBase {
*/ */
protected static MVStore openStore(String fileName) { protected static MVStore openStore(String fileName) {
MVStore store = MVStoreBuilder.fileBased(fileName). MVStore store = MVStoreBuilder.fileBased(fileName).
with(new SampleDataTypeFactory()).open(); with(new SampleTypeFactory()).open();
store.setMaxPageSize(1000); store.setPageSize(1000);
return store; return store;
} }
......
h2/src/test/org/h2/test/store/TestObjectType.java
浏览文件 @ 871b0a8e
...@@ -13,6 +13,7 @@ import java.sql.Timestamp; ...@@ -13,6 +13,7 @@ import java.sql.Timestamp;
import java.util.Arrays; import java.util.Arrays;
import java.util.Random; import java.util.Random;
import java.util.UUID; import java.util.UUID;
import org.h2.dev.store.type.ObjectType;
import org.h2.test.TestBase; import org.h2.test.TestBase;
/** /**
......
h2/src/tools/org/h2/dev/store/btree/MVMap.java
浏览文件 @ 871b0a8e
...@@ -16,6 +16,7 @@ import java.util.List; ...@@ -16,6 +16,7 @@ import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.Set; import java.util.Set;
import java.util.concurrent.ConcurrentMap; import java.util.concurrent.ConcurrentMap;
import org.h2.dev.store.type.DataType;
import org.h2.util.New; import org.h2.util.New;
/** /**
...@@ -72,7 +73,7 @@ public class MVMap<K, V> extends AbstractMap<K, V> ...@@ -72,7 +73,7 @@ public class MVMap<K, V> extends AbstractMap<K, V>
checkWrite(); checkWrite();
long writeVersion = store.getCurrentVersion(); long writeVersion = store.getCurrentVersion();
Page p = root.copyOnWrite(writeVersion); Page p = root.copyOnWrite(writeVersion);
if (p.getMemory() > store.getMaxPageSize() && p.getKeyCount() > 1) { if (p.getMemory() > store.getPageSize() && p.getKeyCount() > 1) {
int at = p.getKeyCount() / 2; int at = p.getKeyCount() / 2;
long totalCount = p.getTotalCount(); long totalCount = p.getTotalCount();
Object k = p.getKey(at); Object k = p.getKey(at);
...@@ -118,7 +119,7 @@ public class MVMap<K, V> extends AbstractMap<K, V> ...@@ -118,7 +119,7 @@ public class MVMap<K, V> extends AbstractMap<K, V>
index++; index++;
} }
Page c = p.getChildPage(index).copyOnWrite(writeVersion); Page c = p.getChildPage(index).copyOnWrite(writeVersion);
if (c.getMemory() > store.getMaxPageSize() && c.getKeyCount() > 1) { if (c.getMemory() > store.getPageSize() && c.getKeyCount() > 1) {
// split on the way down // split on the way down
int at = c.getKeyCount() / 2; int at = c.getKeyCount() / 2;
Object k = c.getKey(at); Object k = c.getKey(at);
...@@ -957,8 +958,12 @@ public class MVMap<K, V> extends AbstractMap<K, V> ...@@ -957,8 +958,12 @@ public class MVMap<K, V> extends AbstractMap<K, V>
DataUtils.appendMap(buff, "name", name); DataUtils.appendMap(buff, "name", name);
DataUtils.appendMap(buff, "type", getType()); DataUtils.appendMap(buff, "type", getType());
DataUtils.appendMap(buff, "createVersion", createVersion); DataUtils.appendMap(buff, "createVersion", createVersion);
DataUtils.appendMap(buff, "key", keyType.asString()); if (keyType != null) {
DataUtils.appendMap(buff, "value", valueType.asString()); DataUtils.appendMap(buff, "key", keyType.asString());
}
if (valueType != null) {
DataUtils.appendMap(buff, "value", valueType.asString());
}
return buff.toString(); return buff.toString();
} }
......
h2/src/tools/org/h2/dev/store/btree/MVStore.java
浏览文件 @ 871b0a8e
...@@ -19,8 +19,12 @@ import java.util.Iterator; ...@@ -19,8 +19,12 @@ import java.util.Iterator;
import java.util.Map; import java.util.Map;
import org.h2.compress.CompressLZF; import org.h2.compress.CompressLZF;
import org.h2.compress.Compressor; import org.h2.compress.Compressor;
import org.h2.dev.store.FilePathCache;
import org.h2.dev.store.cache.CacheLongKeyLIRS; import org.h2.dev.store.cache.CacheLongKeyLIRS;
import org.h2.dev.store.cache.FilePathCache;
import org.h2.dev.store.type.DataType;
import org.h2.dev.store.type.DataTypeFactory;
import org.h2.dev.store.type.ObjectTypeFactory;
import org.h2.dev.store.type.StringType;
import org.h2.store.fs.FilePath; import org.h2.store.fs.FilePath;
import org.h2.store.fs.FileUtils; import org.h2.store.fs.FileUtils;
import org.h2.util.New; import org.h2.util.New;
...@@ -36,6 +40,9 @@ header: ...@@ -36,6 +40,9 @@ header:
H:3,... H:3,...
TODO: TODO:
- support serialization by default
- build script
- test concurrent storing in a background thread
- store store creation in file header, and seconds since creation in chunk header (plus a counter) - store store creation in file header, and seconds since creation in chunk header (plus a counter)
- recovery: keep some old chunks; don't overwritten for 5 minutes (configurable) - recovery: keep some old chunks; don't overwritten for 5 minutes (configurable)
- allocate memory with Utils.newBytes and so on - allocate memory with Utils.newBytes and so on
...@@ -69,9 +76,13 @@ TODO: ...@@ -69,9 +76,13 @@ TODO:
- triggers (can be implemented with a custom map) - triggers (can be implemented with a custom map)
- store write operations per page (maybe defragment if much different than count) - store write operations per page (maybe defragment if much different than count)
- r-tree: nearest neighbor search - r-tree: nearest neighbor search
- use FileChannel by default (nio file system) - use FileChannel by default (nio file system), but: an interrupt close the FileChannel
- auto-save temporary data if it uses too much memory, but revert it on startup if needed. - auto-save temporary data if it uses too much memory, but revert it on startup if needed.
- map and chunk metadata: do not store default values - map and chunk metadata: do not store default values
- support maps without values (non-unique indexes), and maps without keys (counted b-tree)
- use a small object cache (StringCache)
- dump values
- tool to import / manipulate CSV files
*/ */
...@@ -92,7 +103,7 @@ public class MVStore { ...@@ -92,7 +103,7 @@ public class MVStore {
private final String fileName; private final String fileName;
private final DataTypeFactory dataTypeFactory; private final DataTypeFactory dataTypeFactory;
private int maxPageSize = 4 * 1024; private int pageSize = 6 * 1024;
private FileChannel file; private FileChannel file;
private FileLock fileLock; private FileLock fileLock;
...@@ -104,8 +115,7 @@ public class MVStore { ...@@ -104,8 +115,7 @@ public class MVStore {
* split in 16 segments. The stack move distance is 2% of the expected * split in 16 segments. The stack move distance is 2% of the expected
* number of entries. * number of entries.
*/ */
private final CacheLongKeyLIRS<Page> cache = CacheLongKeyLIRS.newInstance( private CacheLongKeyLIRS<Page> cache;
16 * 1024 * 1024, 2048, 16, 16 * 1024 * 1024 / 2048 * 2 / 100);
private int lastChunkId; private int lastChunkId;
private final HashMap<Integer, Chunk> chunks = New.hashMap(); private final HashMap<Integer, Chunk> chunks = New.hashMap();
...@@ -145,12 +155,21 @@ public class MVStore { ...@@ -145,12 +155,21 @@ public class MVStore {
MVStore(HashMap<String, Object> config) { MVStore(HashMap<String, Object> config) {
this.config = config; this.config = config;
this.fileName = (String) config.get("fileName"); this.fileName = (String) config.get("fileName");
this.dataTypeFactory = (DataTypeFactory) config.get("dataTypeFactory"); DataTypeFactory parent = new ObjectTypeFactory();
DataTypeFactory f = (DataTypeFactory) config.get("dataTypeFactory");
if (f == null) {
f = parent;
} else {
f.setParent(parent);
}
this.dataTypeFactory = f;
this.readOnly = "r".equals(config.get("openMode")); this.readOnly = "r".equals(config.get("openMode"));
this.compressor = "0".equals(config.get("compression")) ? null : new CompressLZF(); this.compressor = "0".equals(config.get("compression")) ? null : new CompressLZF();
Object s = config.get("cacheSize"); if (fileName != null) {
if (s != null) { Object s = config.get("cacheSize");
cache.setMaxMemory(Integer.parseInt(s.toString()) * 1024 * 1024); int mb = s == null ? 16 : Integer.parseInt(s.toString());
cache = CacheLongKeyLIRS.newInstance(
mb * 1024 * 1024, 2048, 16, mb * 1024 * 1024 / 2048 * 2 / 100);
} }
} }
...@@ -519,7 +538,7 @@ public class MVStore { ...@@ -519,7 +538,7 @@ public class MVStore {
return currentVersion; return currentVersion;
} }
// the last chunk was not completely correct in the last save() // the last chunk was not completely correct in the last store()
// this needs to be updated now (it's better not to update right after // this needs to be updated now (it's better not to update right after
// storing, because that would modify the meta map again) // storing, because that would modify the meta map again)
Chunk c = chunks.get(lastChunkId); Chunk c = chunks.get(lastChunkId);
...@@ -944,24 +963,24 @@ public class MVStore { ...@@ -944,24 +963,24 @@ public class MVStore {
} }
/** /**
* Set the maximum amount of memory a page should contain, in bytes. Larger * Set the amount of memory a page should contain at most, in bytes. Larger
* pages are split. The default is 4 KB. This is not a limit in the page * pages are split. The default is 6 KB. This is not a limit in the page
* size, as pages with one entry can be larger. As a rule of thumb, pages * size (pages with one entry can get larger), it is just the point where
* should not be larger than 1 MB, for caching to work efficiently. * pages are split.
* *
* @param maxPageSize the page size * @param pageSize the page size
*/ */
public void setMaxPageSize(int maxPageSize) { public void setPageSize(int pageSize) {
this.maxPageSize = maxPageSize; this.pageSize = pageSize;
} }
/** /**
* Get the maximum page size, in bytes. * Get the page size, in bytes.
* *
* @return the maximum page size * @return the page size
*/ */
public int getMaxPageSize() { public int getPageSize() {
return maxPageSize; return pageSize;
} }
Compressor getCompressor() { Compressor getCompressor() {
......
h2/src/tools/org/h2/dev/store/btree/MVStoreBuilder.java
浏览文件 @ 871b0a8e
...@@ -7,6 +7,7 @@ ...@@ -7,6 +7,7 @@
package org.h2.dev.store.btree; package org.h2.dev.store.btree;
import java.util.HashMap; import java.util.HashMap;
import org.h2.dev.store.type.DataTypeFactory;
import org.h2.util.New; import org.h2.util.New;
/** /**
......
h2/src/tools/org/h2/dev/store/btree/MVStoreUtils.java → h2/src/tools/org/h2/dev/store/btree/MVStoreTool.java
浏览文件 @ 871b0a8e
...@@ -19,7 +19,7 @@ import org.h2.store.fs.FileUtils; ...@@ -19,7 +19,7 @@ import org.h2.store.fs.FileUtils;
/** /**
* Utility methods used in combination with the MVStore. * Utility methods used in combination with the MVStore.
*/ */
public class MVStoreUtils { public class MVStoreTool {
/** /**
* Runs this tool. * Runs this tool.
......
h2/src/tools/org/h2/dev/store/btree/Page.java
浏览文件 @ 871b0a8e
...@@ -11,6 +11,7 @@ import java.nio.ByteBuffer; ...@@ -11,6 +11,7 @@ import java.nio.ByteBuffer;
import java.nio.channels.FileChannel; import java.nio.channels.FileChannel;
import java.util.Arrays; import java.util.Arrays;
import org.h2.compress.Compressor; import org.h2.compress.Compressor;
import org.h2.dev.store.type.DataType;
/** /**
* A page (a node or a leaf). * A page (a node or a leaf).
......
h2/src/tools/org/h2/dev/store/cache/CacheLongKeyLIRS.java
浏览文件 @ 871b0a8e
...@@ -148,7 +148,7 @@ public class CacheLongKeyLIRS<V> { ...@@ -148,7 +148,7 @@ public class CacheLongKeyLIRS<V> {
* @param key the key (may not be null) * @param key the key (may not be null)
* @return the old value, or null if there was no resident entry * @return the old value, or null if there was no resident entry
*/ */
public synchronized V remove(long key) { public V remove(long key) {
int hash = getHash(key); int hash = getHash(key);
return getSegment(hash).remove(key, hash); return getSegment(hash).remove(key, hash);
} }
......
h2/src/tools/org/h2/dev/store/FilePathCache.java → h2/src/tools/org/h2/dev/store/cache/FilePathCache.java
浏览文件 @ 871b0a8e
...@@ -4,13 +4,12 @@ ...@@ -4,13 +4,12 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.dev.store; package org.h2.dev.store.cache;
import java.io.IOException; import java.io.IOException;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import java.nio.channels.FileChannel; import java.nio.channels.FileChannel;
import java.nio.channels.FileLock; import java.nio.channels.FileLock;
import org.h2.dev.store.cache.CacheLongKeyLIRS;
import org.h2.store.fs.FileBase; import org.h2.store.fs.FileBase;
import org.h2.store.fs.FilePathWrapper; import org.h2.store.fs.FilePathWrapper;
......
h2/src/test/org/h2/test/store/MVRTreeMap.java → h2/src/tools/org/h2/dev/store/rtree/MVRTreeMap.java
浏览文件 @ 871b0a8e
...@@ -4,15 +4,15 @@ ...@@ -4,15 +4,15 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.test.store; package org.h2.dev.store.rtree;
import java.util.ArrayList; import java.util.ArrayList;
import java.util.Iterator; import java.util.Iterator;
import org.h2.dev.store.btree.Cursor; import org.h2.dev.store.btree.Cursor;
import org.h2.dev.store.btree.CursorPos; import org.h2.dev.store.btree.CursorPos;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.MVMap; import org.h2.dev.store.btree.MVMap;
import org.h2.dev.store.btree.Page; import org.h2.dev.store.btree.Page;
import org.h2.dev.store.type.DataType;
import org.h2.util.New; import org.h2.util.New;
/** /**
...@@ -50,7 +50,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> { ...@@ -50,7 +50,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> {
checkOpen(); checkOpen();
return new RTreeCursor(this, root, x) { return new RTreeCursor(this, root, x) {
protected boolean check(boolean leaf, SpatialKey key, SpatialKey test) { protected boolean check(boolean leaf, SpatialKey key, SpatialKey test) {
return keyType.contains(key, test); return keyType.isOverlap(key, test);
} }
}; };
} }
...@@ -201,7 +201,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> { ...@@ -201,7 +201,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> {
Page p = root.copyOnWrite(writeVersion); Page p = root.copyOnWrite(writeVersion);
Object result; Object result;
if (alwaysAdd || get(key) == null) { if (alwaysAdd || get(key) == null) {
if (p.getMemory() > store.getMaxPageSize() && p.getKeyCount() > 1) { if (p.getMemory() > store.getPageSize() && p.getKeyCount() > 1) {
// only possible if this is the root, else we would have split earlier // only possible if this is the root, else we would have split earlier
// (this requires maxPageSize is fixed) // (this requires maxPageSize is fixed)
long totalCount = p.getTotalCount(); long totalCount = p.getTotalCount();
...@@ -283,7 +283,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> { ...@@ -283,7 +283,7 @@ public class MVRTreeMap<V> extends MVMap<SpatialKey, V> {
} }
} }
Page c = p.getChildPage(index).copyOnWrite(writeVersion); Page c = p.getChildPage(index).copyOnWrite(writeVersion);
if (c.getMemory() > store.getMaxPageSize() && c.getKeyCount() > 1) { if (c.getMemory() > store.getPageSize() && c.getKeyCount() > 1) {
// split on the way down // split on the way down
Page split = split(c, writeVersion); Page split = split(c, writeVersion);
p = p.copyOnWrite(writeVersion); p = p.copyOnWrite(writeVersion);
......
h2/src/test/org/h2/test/store/SpatialKey.java → h2/src/tools/org/h2/dev/store/rtree/SpatialKey.java
浏览文件 @ 871b0a8e
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.test.store; package org.h2.dev.store.rtree;
import java.util.Arrays; import java.util.Arrays;
......
h2/src/test/org/h2/test/store/SpatialType.java → h2/src/tools/org/h2/dev/store/rtree/SpatialType.java
浏览文件 @ 871b0a8e
...@@ -4,12 +4,12 @@ ...@@ -4,12 +4,12 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.test.store; package org.h2.dev.store.rtree;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import java.util.ArrayList; import java.util.ArrayList;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.DataUtils; import org.h2.dev.store.btree.DataUtils;
import org.h2.dev.store.type.DataType;
/** /**
* A spatial data type. This class supports up to 255 dimensions. Each dimension * A spatial data type. This class supports up to 255 dimensions. Each dimension
......
h2/src/tools/org/h2/dev/store/btree/DataType.java → h2/src/tools/org/h2/dev/store/type/DataType.java
浏览文件 @ 871b0a8e
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.dev.store.btree; package org.h2.dev.store.type;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
...@@ -58,6 +58,9 @@ public interface DataType { ...@@ -58,6 +58,9 @@ public interface DataType {
/** /**
* Get the stable string representation that is used to build this data * Get the stable string representation that is used to build this data
* type. * type.
* <p>
* To avoid conflict with the default factory, the returned string should
* start with the package name of the type factory.
* *
* @return the string representation * @return the string representation
*/ */
......
h2/src/tools/org/h2/dev/store/btree/DataTypeFactory.java → h2/src/tools/org/h2/dev/store/type/DataTypeFactory.java
浏览文件 @ 871b0a8e
...@@ -4,27 +4,36 @@ ...@@ -4,27 +4,36 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.dev.store.btree; package org.h2.dev.store.type;
/** /**
* A factory for maps and data types. * A factory for maps and data types.
*/ */
public interface DataTypeFactory { public interface DataTypeFactory {
/**
* Set the parent factory.
*
* @param parent the parent factory
*/
void setParent(DataTypeFactory parent);
/** /**
* Parse the data type. * Parse the data type.
* *
* @param dataType the string and type specific meta data * @param dataType the string and type specific meta data
* @return the type * @return the type, or null if unknown
*/ */
DataType buildDataType(String dataType); DataType buildDataType(String dataType);
/** /**
* Get the data type object for the given class. * Get the data type identifier for the given class.
* <p>
* To avoid conflict with the default factory, the returned string should
* start with the package name of the type factory.
* *
* @param objectClass the class * @param objectClass the class
* @return the data type object * @return the data type identifier, or null if not supported
*/ */
String getDataType(Class<?> objectClass); String getDataType(Class<?> objectClass);
......
h2/src/test/org/h2/test/store/ObjectType.java → h2/src/tools/org/h2/dev/store/type/ObjectType.java
浏览文件 @ 871b0a8e
...@@ -4,13 +4,12 @@ ...@@ -4,13 +4,12 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.test.store; package org.h2.dev.store.type;
import java.math.BigDecimal; import java.math.BigDecimal;
import java.math.BigInteger; import java.math.BigInteger;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import java.util.UUID; import java.util.UUID;
import org.h2.dev.store.btree.DataType;
import org.h2.dev.store.btree.DataUtils; import org.h2.dev.store.btree.DataUtils;
import org.h2.util.Utils; import org.h2.util.Utils;
...@@ -652,7 +651,7 @@ public class ObjectType implements DataType { ...@@ -652,7 +651,7 @@ public class ObjectType implements DataType {
/** /**
* The type for long objects. * The type for long objects.
*/ */
class LongType extends AutoDetectDataType { public class LongType extends AutoDetectDataType {
LongType(ObjectType base) { LongType(ObjectType base) {
super(base, TYPE_LONG); super(base, TYPE_LONG);
......
h2/src/test/org/h2/test/store/SampleDataTypeFactory.java → h2/src/tools/org/h2/dev/store/type/ObjectTypeFactory.java
浏览文件 @ 871b0a8e
...@@ -3,43 +3,36 @@ ...@@ -3,43 +3,36 @@
* 1.0, and under the Eclipse Public License, Version 1.0 * 1.0, and under the Eclipse Public License, Version 1.0
* (http://h2database.com/html/license.html). Initial Developer: H2 Group * (http://h2database.com/html/license.html). Initial Developer: H2 Group
*/ */
package org.h2.test.store; package org.h2.dev.store.type;
import org.h2.dev.store.btree.DataType; import org.h2.dev.store.rtree.SpatialType;
import org.h2.dev.store.btree.DataTypeFactory;
import org.h2.dev.store.btree.StringType;
/** /**
* A data type factory. * A data type factory.
*/ */
public class SampleDataTypeFactory implements DataTypeFactory { public class ObjectTypeFactory implements DataTypeFactory {
@Override
public void setParent(DataTypeFactory parent) {
// never called for this factory
}
@Override @Override
public DataType buildDataType(String s) { public DataType buildDataType(String s) {
if (s.length() == 0) { if ("s".equals(s)) {
return new StringType();
}
switch (s.charAt(0)) {
case 'i':
return new IntegerType();
case 'r':
return RowType.fromString(s, this);
case 's':
return SpatialType.fromString(s); return SpatialType.fromString(s);
case 'o': } else if ("o".equals(s)) {
return new ObjectType(); return new ObjectType();
} }
throw new RuntimeException("Unknown data type " + s); return null;
} }
@Override @Override
public String getDataType(Class<?> objectClass) { public String getDataType(Class<?> objectClass) {
if (objectClass == Integer.class) { if (objectClass == SpatialType.class) {
return "i"; return "s";
} else if (Object.class == Object.class) {
return "o";
} }
throw new RuntimeException("Unsupported object class " + objectClass.toString()); return "o";
} }
} }
h2/src/tools/org/h2/dev/store/btree/StringType.java → h2/src/tools/org/h2/dev/store/type/StringType.java
浏览文件 @ 871b0a8e
...@@ -4,9 +4,10 @@ ...@@ -4,9 +4,10 @@
* (http://h2database.com/html/license.html). * (http://h2database.com/html/license.html).
* Initial Developer: H2 Group * Initial Developer: H2 Group
*/ */
package org.h2.dev.store.btree; package org.h2.dev.store.type;
import java.nio.ByteBuffer; import java.nio.ByteBuffer;
import org.h2.dev.store.btree.DataUtils;
/** /**
* A string type. * A string type.
......
Markdown 格式
0% 或
您添加了 0 人 到此讨论。请谨慎行事。
请先完成此评论的编辑！
请 注册 或者 后发表评论