-
Notifications
You must be signed in to change notification settings - Fork 128
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Loading status checks…
MAJOR RELEASE: VERSION 1.0. BREAKS BACKWARD COMPATABILITYgit add . SE…
…E README.md FOR COMPLETE CHANGELOG
patx
committed
Jan 10, 2025
1 parent
2dd930d
commit d01baec
Showing
10 changed files
with
553 additions
and
1,190 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,196 +1,249 @@ | ||
 | ||
# **pickleDB: Your Lightweight, High-Speed Key-Value Store** | ||
|
||
# pickleDB | ||
pickleDB is lightweight, fast, and simple database based on the | ||
[json](https://docs.python.org/3/library/json.html) module. | ||
And it's BSD licensed! | ||
## **Fast. Simple. Reliable.** | ||
Unlock the power of effortless data storage with **pickleDB**—the no-fuss, blazing-fast key-value store designed for Python developers. Whether you're building a small script or a performant microservice, pickleDB delivers simplicity and speed with the reliability you can count on. | ||
|
||
--- | ||
|
||
## pickleDB is Fun | ||
```python | ||
>>> import pickledb | ||
## **Why Choose pickleDB?** | ||
|
||
>>> db = pickledb.load('test.db', False) | ||
### ✅ **Blazing Speed** | ||
Backed by the high-performance [orjson](https://pypi.org/project/orjson/) library, pickleDB handles millions of records with ease. Perfect for applications where every millisecond counts. | ||
|
||
>>> db.set('key', 'value') | ||
### ✅ **Ridiculously Easy to Use** | ||
With its minimalist API, pickleDB makes adding, retrieving, and managing your data as simple as writing a Python list. No steep learning curves. No unnecessary complexity. | ||
|
||
>>> db.get('key') | ||
'value' | ||
### ✅ **Rock-Solid Reliability** | ||
Your data deserves to be safe. Atomic saves ensure your database remains consistent—even if something goes wrong. | ||
|
||
>>> db.dump() | ||
True | ||
``` | ||
### ✅ **Pythonic Flexibility** | ||
Store strings, lists, dictionaries, and more—all with native Python operations. No need to learn special commands. If you know Python, you already know pickleDB. | ||
|
||
## Installation | ||
- Just add the `pickledb.py` file to your working directory or use `pip install pickledb`. | ||
- pickleDB also includes a simplified and faster version using orjson called `pkldb.py`, to use this add it to your working directory and note the slight difference in setup `from pkldb.py import pkldb` then `db = pkldb('example.json')` | ||
--- | ||
|
||
# PickleDB Documentation | ||
## **Getting Started** | ||
|
||
## Introduction | ||
PickleDB is a lightweight, file-based key-value store with optional support for time-to-live (TTL). It provides a simple and intuitive API for storing and managing data persistently. | ||
### **Install in Seconds** | ||
pickleDB is available on PyPI. Get started with just one command: | ||
```bash | ||
pip install pickledb | ||
``` | ||
|
||
--- | ||
### **Your First pickleDB** | ||
```python | ||
from pickledb import PickleDB | ||
|
||
## Table of Contents | ||
1. **Basic Usage** | ||
2. **Key-Value Methods** | ||
3. **List Methods** | ||
4. **Dictionary Methods** | ||
5. **Enhanced Features** | ||
# Initialize the database | ||
db = PickleDB('my_database.db') | ||
|
||
--- | ||
# Add a key-value pair | ||
db.set('greeting', 'Hello, world!') | ||
|
||
## 1. Basic Usage | ||
```python | ||
from pickledb_enhanced import load | ||
# Retrieve the value | ||
print(db.get('greeting')) # Output: Hello, world! | ||
|
||
db = load('mydb.json', auto_dump=True, enable_ttl=True) | ||
# Save the data to disk | ||
db.save() | ||
``` | ||
- `auto_dump`: Automatically save changes to the file. | ||
- `enable_ttl`: Enable TTL support for expiring keys. | ||
It’s that simple! In just a few lines, you have a fully functioning key-value store. | ||
|
||
--- | ||
|
||
## 2. Key-Value Methods | ||
## **More Examples to Get You Inspired** | ||
|
||
### `set(key, value, ttl=None)` | ||
Set a key-value pair in the database. | ||
|
||
### `get(key)` | ||
Retrieve the value associated with a key. | ||
### **Store and Retrieve Complex Data** | ||
PickleDB works seamlessly with Python data structures. Example: | ||
```python | ||
# Store a dictionary | ||
db.set('user', {'name': 'Alice', 'age': 30, 'city': 'Wonderland'}) | ||
|
||
### `exists(key)` | ||
Check if a key exists. | ||
# Retrieve and update it | ||
user = db.get('user') | ||
user['age'] += 1 | ||
|
||
### `rem(key)` | ||
Remove a key from the database. | ||
# Save the updated data | ||
db.set('user', user) | ||
print(db.get('user')) # Output: {'name': 'Alice', 'age': 31, 'city': 'Wonderland'} | ||
``` | ||
|
||
### `getall()` | ||
Get all keys in the database. | ||
### **Use Lists for Dynamic Data** | ||
Handle lists with ease: | ||
```python | ||
# Add a list of items | ||
db.set('tasks', ['Write code', 'Test app', 'Deploy']) | ||
|
||
### `clear()` | ||
Clear all keys. | ||
# Retrieve and modify | ||
tasks = db.get('tasks') | ||
tasks.append('Celebrate') | ||
db.set('tasks', tasks) | ||
|
||
### `deldb()` | ||
Delete the database file. | ||
print(db.get('tasks')) # Output: ['Write code', 'Test app', 'Deploy', 'Celebrate'] | ||
``` | ||
|
||
--- | ||
### **Store Configurations** | ||
Create a simple, persistent configuration store: | ||
```python | ||
# Set configuration options | ||
db.set('config', {'theme': 'dark', 'notifications': True}) | ||
|
||
# Access and update settings | ||
config = db.get('config') | ||
config['notifications'] = False | ||
db.set('config', config) | ||
print(db.get('config')) # Output: {'theme': 'dark', 'notifications': False} | ||
``` | ||
|
||
## 3. List Methods | ||
### **Session Management** | ||
Track user sessions effortlessly: | ||
```python | ||
# Add session data | ||
db.set('session_12345', {'user_id': 1, 'status': 'active'}) | ||
|
||
### `lcreate(name)` | ||
Create a new list in the database. | ||
# End a session | ||
session = db.get('session_12345') | ||
session['status'] = 'inactive' | ||
db.set('session_12345', session) | ||
|
||
### `ladd(name, value)` | ||
Add a value to an existing list. | ||
print(db.get('session_12345')) # Output: {'user_id': 1, 'status': 'inactive'} | ||
``` | ||
|
||
### `lgetall(name)` | ||
Retrieve all values from a list. | ||
--- | ||
|
||
### `lsort(name, reverse=False)` | ||
Sort a list in ascending or descending order. | ||
- `reverse`: Sort in descending order if `True`. | ||
## **Performance Highlights** | ||
|
||
### `lremove(name, value)` | ||
Remove a value from a list. | ||
pickleDB demonstrates strong performance for handling large-sized datasets: | ||
|
||
### `lgetrange(name, start, end)` | ||
Retrieve a range of values from a list. | ||
- `start`: Start index. | ||
- `end`: End index. | ||
| Entries | Memory Load Time | Retrieval Time | Save Time | | ||
|--------------|------------------|----------------|-----------| | ||
| **1M** | 1.21 sec | 0.90 sec | 0.17 sec | | ||
| **10M** | 14.11 sec | 10.30 sec | 1.67 sec | | ||
| **50M** | 93.79 sec | 136.42 sec | 61.08 sec | | ||
|
||
### `llen(name)` | ||
Get the length of a list. | ||
Tests were performed on a StarLabs StarLite Mk IV (Quad-Core Intel® Pentium® Silver N5030 CPU @ 1.10GHz w/ 8GB memory) running elementary OS 7.1 Horus. | ||
|
||
--- | ||
|
||
## 4. Dictionary Methods | ||
## **Minimal, Powerful API** | ||
|
||
pickleDB offers a clean and Pythonic API for managing data efficiently: | ||
|
||
### `dcreate(name)` | ||
Create a new dictionary in the database. | ||
### **`set(key, value)`** | ||
Add or update a key-value pair: | ||
```python | ||
# Add a new key-value pair | ||
db.set('username', 'admin') | ||
|
||
### `dadd(name, key, value)` | ||
Add a key-value pair to a dictionary. | ||
# Update an existing key-value pair | ||
db.set('username', 'superadmin') | ||
print(db.get('username')) # Output: 'superadmin' | ||
``` | ||
|
||
### `dget(name, key)` | ||
Retrieve a value from a dictionary. | ||
### **`get(key)`** | ||
Retrieve the value associated with a key: | ||
```python | ||
# Get the value for a key | ||
print(db.get('username')) # Output: 'superadmin' | ||
|
||
### `dgetall(name)` | ||
Retrieve all key-value pairs from a dictionary. | ||
# Attempt to retrieve a non-existent key | ||
print(db.get('nonexistent_key')) # Output: None | ||
``` | ||
|
||
### `dremove(name, key)` | ||
Remove a key from a dictionary. | ||
### **`all()`** | ||
Get a list of all keys: | ||
```python | ||
# Add multiple keys | ||
db.set('item1', 'value1') | ||
db.set('item2', 'value2') | ||
|
||
### `dmerge(name, other_dict)` | ||
Merge another dictionary into an existing dictionary. | ||
# Retrieve all keys | ||
print(db.all()) # Output: ['username', 'item1', 'item2'] | ||
``` | ||
|
||
### `dkeys(name)` | ||
Get all keys from a dictionary. | ||
### **`remove(key)`** | ||
Delete a key and its value: | ||
```python | ||
# Remove a key-value pair | ||
db.remove('item1') | ||
print(db.all()) # Output: ['username', 'item2'] | ||
``` | ||
|
||
### `dvalues(name)` | ||
Get all values from a dictionary. | ||
### **`purge()`** | ||
Clear all data in the database: | ||
```python | ||
# Clear the database | ||
db.purge() | ||
print(db.all()) # Output: [] | ||
``` | ||
|
||
### **`save()`** | ||
Persist the database to disk: | ||
```python | ||
# Save the current state of the database | ||
db.save() | ||
print("Database saved successfully!") | ||
``` | ||
|
||
--- | ||
|
||
## 5. Enhanced Features | ||
## **Key Improvements in Version 1.0** | ||
|
||
### **TTL Support** | ||
- Expire keys automatically after a given time. | ||
pickleDB 1.0 is a reimagined version designed for speed, simplicity, and reliability. Key changes include: | ||
|
||
### **File Compression** | ||
- Compress the database file to save space. | ||
- **Atomic Saves**: Ensures data integrity during writes, eliminating potential corruption issues. | ||
- **Faster Serialization**: Switched to `orjson` for significantly improved speed. | ||
- **Streamlined API**: Removed legacy methods (e.g., `ladd`, `dmerge`) in favor of native Python operations. | ||
- **Unified Handling of Data Types**: Treats all Python-native types (lists, dicts, etc.) as first-class citizens. | ||
- **Explicit Saves**: The `auto_save` feature was removed to provide users greater control and optimize performance. | ||
|
||
### **Automatic Persistence** | ||
- Save changes automatically using `auto_dump`. | ||
If backward compatibility is essential, version 0.9 is still available: | ||
- View the legacy code [here](https://gist.github.com/patx/3ad47fc3814d7293feb902f6ab49c48f). | ||
- Install it by: | ||
```bash | ||
pip uninstall pickledb | ||
``` | ||
Then download the legacy file and include it in your project. | ||
|
||
--- | ||
|
||
## Example Usage | ||
## **Limitations** | ||
|
||
### **Working with Lists** | ||
```python | ||
# Create a list and add values | ||
db.lcreate('mylist') | ||
db.ladd('mylist', 'item1') | ||
db.ladd('mylist', 'item2') | ||
While pickleDB is powerful, it’s important to understand its limitations: | ||
|
||
# Sort the list | ||
db.lsort('mylist') # ['item1', 'item2'] | ||
- **Memory Usage**: The entire dataset is loaded into memory, which might be a constraint on systems with limited RAM for extremely large datasets. | ||
- **Single-Threaded**: The program is not thread-safe. For concurrent access, use external synchronization like Python's `RLock()`. | ||
- **Blocking Saves**: Saves are blocking by default. To achieve non-blocking saves, use asynchronous wrappers. | ||
- **Lack of Advanced Features**: pickleDB is designed for simplicity, so it may not meet the needs of applications requiring advanced database features. | ||
|
||
# Get a range of values | ||
db.lgetrange('mylist', 0, 1) # ['item1'] | ||
For projects requiring more robust solutions, consider alternatives like **[kenobiDB](Https://github.com/patx/kenobi)**, [Redis](http://redis.io/), [SQLite](https://www.sqlite.org/), or [MongoDB](https://www.mongodb.com/). | ||
|
||
# Remove an item | ||
db.lremove('mylist', 'item1') | ||
``` | ||
--- | ||
|
||
### **Working with Dictionaries** | ||
```python | ||
# Create a dictionary and add values | ||
db.dcreate('mydict') | ||
db.dadd('mydict', 'key1', 'value1') | ||
db.dadd('mydict', 'key2', 'value2') | ||
## **Asynchronous Saves** | ||
Want non-blocking saves? You can implement an async wrapper to handle saves in the background. This is particularly useful for applications that need high responsiveness without delaying due to disk operations, like small web applications. Check out examples [here](https://gist.github.com/patx/5c12d495ff142f3262325eeae81eb000). | ||
|
||
# Merge another dictionary | ||
db.dmerge('mydict', {'key3': 'value3'}) | ||
--- | ||
|
||
# Get all keys and values | ||
db.dkeys('mydict') # ['key1', 'key2', 'key3'] | ||
db.dvalues('mydict') # ['value1', 'value2', 'value3'] | ||
## **Community & Contributions** | ||
|
||
# Remove a key | ||
db.dremove('mydict', 'key1') | ||
``` | ||
### **Join the Community** | ||
We’re passionate about making pickleDB better every day. Got ideas, feedback, or an issue to report? Let’s connect: | ||
- **File an Issue**: [GitHub Issues](https://github.com/patx/pickledb/issues) | ||
- **Ask Questions**: Reach out to our growing community of users and developers. | ||
|
||
--- | ||
### **Contribute to pickleDB** | ||
Want to leave your mark? Help us make pickleDB even better: | ||
- **Submit a Pull Request**: Whether it's fixing a bug, improving the documentation, or adding a feature, we’d love your contributions. | ||
- **Suggest New Features**: Share your ideas to make pickleDB more powerful. | ||
|
||
## Notes | ||
- Always ensure proper file permissions for the database file. | ||
- Use thread-safe practices when accessing the database concurrently. | ||
Together, we can build a better tool for everyone. | ||
|
||
--- | ||
|
||
## Changelog | ||
- **Enhanced Features**: Added methods for list sorting, removal, range fetching, and dictionary merging. | ||
## **Documentation** | ||
|
||
Explore the full capabilities of pickleDB with our detailed documentation: | ||
- **API Reference**: [Commands and Examples](https://patx.github.io/pickledb/commands.html) | ||
- [GitHub Repository](https://github.com/patx/pickledb) | ||
- [Installation Details (PyPI)](http://pypi.python.org/pypi/pickleDB) | ||
|
||
Whether you're a beginner or an experienced developer, these resources will guide you through everything pickleDB has to offer. |
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,59 +1,82 @@ | ||
|
||
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> | ||
<html> | ||
<!DOCTYPE html> | ||
<html lang="en"> | ||
<head> | ||
<link rel="shortcut icon" href="http://packages.python.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<link rel="icon" href="http://packages.python.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> | ||
<title>pickleDB - simple key-value database</title> | ||
<style type="text/css"> | ||
.logo { width: 800px; margin: 40px auto; } | ||
.body { font-family: "helvetica"; width: 540px; margin: 40px auto; } | ||
img.c1 { position: absolute; top: 0; right: 0; border: 0; } | ||
span.c2 { color: #8F5902 } | ||
span.c9 { color: #4E9A06 } | ||
a { color: #008000; border-bottom: 0px dotted white; text-decoration: none; } | ||
a:hover { color: #008000; border-bottom: 0px solid white; } | ||
</style> | ||
<meta charset="utf-8"> | ||
<meta name="viewport" content="width=device-width, initial-scale=1.0"> | ||
<link rel="shortcut icon" href="https://packages.python.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<link rel="icon" href="https://packages.python.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<title>pickleDB API Documentation</title> | ||
<style> | ||
.logo { width: 800px; margin: 40px auto; } | ||
.body { font-family: "Helvetica", sans-serif; width: 540px; margin: 40px auto; } | ||
img.c1 { position: absolute; top: 0; right: 0; border: 0; } | ||
span.c2 { color: #8F5902; } | ||
span.c9 { color: #4E9A06; } | ||
a { color: #008000; border-bottom: 0px dotted white; text-decoration: none; } | ||
a:hover { color: #008000; border-bottom: 0px solid white; } | ||
</style> | ||
</head> | ||
<body> | ||
<div class="logo"><a href="index.html"><img src="logo.png" alt="pickleDB logo"></a></div> | ||
<div class="body"> | ||
<h1>Current Commands</h1> | ||
<p><code><span class="c2">LOAD</span> <span class="c9">path</span> <span class="c9">auto_dump</span></code> → Load a database from a path with auto_dump enabled or not <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">SET</span> <span class="c9">key</span> <span class="c9">value</span></code> → Set the value of a <em>str</em> key <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">GET</span> <span class="c9">key</span></code> → Get the value of a key <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">GETALL</span></code> → Return a list of all keys in database <small><em>(available since 0.4)</em></small></p> | ||
<p><code><span class="c2">REM</span> <span class="c9">key</span></code> → Delete a key <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">APPEND</span> <span class="c9">key</span> <span class="c9">more</span></code> → Add more to a key's value <small><em>(available since 0.1.3)</em></small></p> | ||
<p><code><span class="c2">EXISTS</span> <span class="c9">key</span></code> → Determine if a key exists <small><em>(available since 0.7.2)</em></small></p> | ||
<p><code><span class="c2">TOTALKEYS</span> <span class="c9">name</span></code> → Get the total number of keys in whole database or in a specified (<code>name</code>) dict or list <small><em>(available since 0.7.3)</em></small></p> | ||
<p><code><span class="c2">LCREATE</span> <span class="c9">name</span></code> → Create a list with <em>str</em> name <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LADD</span> <span class="c9">name</span> <span class="c9">value</span></code> → Add a value to a list <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LGETALL</span> <span class="c9">name</span></code> → Return all values in a list <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LEXTEND</span> <span class="c9">name</span> <span class="c9">seq</span></code> → Extend a list with a sequence <small><em>(available since 0.6)</em></small></p> | ||
<p><code><span class="c2">LGET</span> <span class="c9">name</span> <span class="c9">pos</span></code> → Return one value in a list <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LRANGE</span> <span class="c9">name</span> <span class="c9">start</span> <span class="c9">end</span></code> → Return all the values from a given range in a list <small><em></em></small></p> | ||
<p><code><span class="c2">LREMLIST</span> <span class="c9">name</span></code> → Remove a list and all of its values <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LREMVALUE</span> <span class="c9">name</span> <span class="c9">value</span></code> → Remove a value from list <code>name</code> <small><em>(available since 0.8.2)</em></small></p> | ||
<p><code><span class="c2">LPOP</span> <span class="c9">name</span> <span class="c9">pos</span></code> → Remove one value in a list <small><em>(available since 0.1)</em></small></p> | ||
<p><code><span class="c2">LLEN</span> <span class="c9">name</span></code> → Return the length of a list <small><em>(available since 0.6)</em></small></p> | ||
<p><code><span class="c2">LAPPEND</span> <span class="c9">name</span> <span class="c9">pos</span> <span class="c9">more</span></code> → Add more to a value in a list <small><em>(available since 0.1.3)</em></small></p> | ||
<p><code><span class="c2">LEXISTS</span> <span class="c9">name</span> <span class="c9">value</span></code> → Determine if a value is in a certain list <small><em>(available since 0.7.2)</em></small></p> | ||
<p><code><span class="c2">DCREATE</span> <span class="c9">name</span></code> → Create a dict with <em>str</em> name <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DADD</span> <span class="c9">name</span> <span class="c9">pair</span></code> → Add a key-value pair to a dict, <code>pair</code> is a tuple <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DGETALL</span> <span class="c9">name</span></code> → Return all key-value pairs from a dict <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DGET</span> <span class="c9">name</span> <span class="c9">key</span></code> → Return the value for a key in a dict <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DKEYS</span> <span class="c9">name</span></code> → Return all the keys for a dict <small><em>(available since 0.6)</em></small></p> | ||
<p><code><span class="c2">DVALS</span> <span class="c9">name</span></code> → Return all the values for a dict <small><em>(available) since 0.6)</em></small></p> | ||
<p><code><span class="c2">DEXISTS</span> <span class="c9">name</span> <span class="c9">key</span></code> → Determine if a key exists <small><em>(available since 0.6)</em></small></p> | ||
<p><code><span class="c2">DREM</span> <span class="c9">name</span></code> → Remove a dict and all of its pairs <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DPOP</span> <span class="c9">name</span> <span class="c9">key</span></code> → Remove one key-value in a dict <small><em>(available since 0.2.2)</em></small></p> | ||
<p><code><span class="c2">DMERGE</span> <span class="c9">name1</span> <span class="c9">name2</span> <span class="c9">name3</span></code> → Merge <code>name1</code> and <code>name2</code> into a new dict: <code>name3</code> <small><em>(available since 0.7.3)</em></small> | ||
<p><code><span class="c2">DELDB</span></code> → Delete everything from the database <small><em>(available since 0.2.1)</em></small></p> | ||
<p><code><span class="c2">DUMP</span></code> → Save the database from memory to a file specified in <code>LOAD</code> <small><em>(available since 0.3)</em></small></p> | ||
<h1>Suggestions</h1> | ||
<p>If you would like to suggest a command, you can create an <a href="http://github.com/patx/pickledb/issues">issue on GitHub</a>.</p> | ||
</div> | ||
<div class="logo"> | ||
<a href="index.html"> | ||
<img src="logo.png" alt="pickleDB logo"> | ||
</a> | ||
</div> | ||
<div class="body"> | ||
<h1>API Documentation</h1> | ||
<p>For examples and help on getting started/installation see the <a href="https://github.com/patx/pickledb/?tab=readme-ov-file#readme">README on GitHub.</a></p> | ||
|
||
<p> | ||
<strong>Class Initialization</strong> | ||
</p> | ||
|
||
<p><code><span class="c2">PickleDB</span>(<span class="c9">path</span>)</code></p> | ||
<p>Initialize a PickleDB instance with the specified path.</p> | ||
<ul> | ||
<li><span class="c9">path</span>: The path to the database file.</li> | ||
</ul> | ||
|
||
<p> | ||
<strong><code><span class="c2">PickleDB</span></code> Class Methods</strong> | ||
</p> | ||
|
||
<p><code><span class="c2">set</span>(<span class="c9">key</span>, <span class="c9">value</span>)</code> → Add or update a key-value pair in the database.</p> | ||
<ul> | ||
<li><span class="c9">key</span>: The key to set. Converted to string if not already.</li> | ||
<li><span class="c9">value</span>: The value to associate with the key. This can be any JSON serializable Python data type.</li> | ||
<li><span class="c9">Returns</span>: <em>True</em>.</li> | ||
</ul> | ||
|
||
<p><code><span class="c2">get</span>(<span class="c9">key</span>)</code> → Retrieve the value associated with a key.</p> | ||
<ul> | ||
<li><span class="c9">key</span>: The key to retrieve.</li> | ||
<li><span class="c9">Returns</span>: The value associated with the key, or <em>None</em> if the key does not exist.</li> | ||
</ul> | ||
|
||
<p><code><span class="c2">all</span>()</code> → Retrieve a list of all keys in the database.</p> | ||
<ul> | ||
<li><span class="c9">Returns</span>: A list of keys.</li> | ||
</ul> | ||
|
||
<p><code><span class="c2">remove</span>(<span class="c9">key</span>)</code> → Delete a key and its value from the database.</p> | ||
<ul> | ||
<li><span class="c9">key</span>: The key to delete.</li> | ||
<li><span class="c9">Returns</span>: <em>True</em> if the key was deleted, or <em>False</em> if the key does not exist.</li> | ||
</ul> | ||
|
||
<p><code><span class="c2">purge</span>()</code> → Clear all keys and values from the database.</p> | ||
<ul> | ||
<li><span class="c9">Returns</span>: <em>True</em>.</li> | ||
</ul> | ||
|
||
<p><code><span class="c2">save</span>()</code> → Save the current state of the database to the file.</p> | ||
<ul> | ||
<li><span class="c9">Returns</span>: <em>True</em> if the operation succeeds, or <em>False</em> otherwise.</li> | ||
<li>For larger datasets consider using a non-blocking wrapper around this method <a href="https://gist.github.com/patx/5c12d495ff142f3262325eeae81eb000">like this</a>. | ||
</ul> | ||
|
||
<h1>Suggestions</h1> | ||
<p>If you would like to suggest an improvement or report an issue, please create an <a href="https://github.com/patx/pickledb/issues">issue on GitHub</a>.</p> | ||
</div> | ||
</body> | ||
</html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,66 +1,82 @@ | ||
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> | ||
<html> | ||
<!DOCTYPE html> | ||
<html lang="en"> | ||
<head> | ||
<link rel="shortcut icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<link rel="icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"> | ||
<title>pickleDB - simple key-value database</title> | ||
|
||
<style type="text/css"> | ||
.logo { width: 800px; margin: 40px auto; } | ||
.body { font-family: "helvetica"; width: 540px; margin: 40px auto; } | ||
img.c1 { position: absolute; top: 0; right: 0; border: 0; } | ||
span.c2 { color: #8F5902 } | ||
span.c9 { color: #4E9A06 } | ||
a { color: #008000; border-bottom: 1px dotted #008000; text-decoration: none; } | ||
a:hover { color: #008000; border-bottom: 1px solid #008000; } | ||
</style> | ||
<meta charset="UTF-8"> | ||
<meta name="viewport" content="width=device-width, initial-scale=1.0"> | ||
<link rel="shortcut icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<link rel="icon" href="https://pythonhosted.org/pickleDB/favicon.ico" type="image/x-icon"> | ||
<title>pickleDB - Simple Key-Value Database</title> | ||
<style> | ||
.logo { | ||
width: 800px; | ||
margin: 40px auto; | ||
} | ||
.body { | ||
font-family: "Helvetica", sans-serif; | ||
width: 540px; | ||
margin: 40px auto; | ||
} | ||
img.c1 { | ||
position: absolute; | ||
top: 0; | ||
right: 0; | ||
border: 0; | ||
} | ||
span.c2 { | ||
color: #8F5902; | ||
} | ||
span.c9 { | ||
color: #4E9A06; | ||
} | ||
a { | ||
color: #008000; | ||
border-bottom: 1px dotted #008000; | ||
text-decoration: none; | ||
} | ||
a:hover { | ||
color: #008000; | ||
border-bottom: 1px solid #008000; | ||
} | ||
</style> | ||
</head> | ||
<body> | ||
<div class="logo"><img src="logo.png" alt="pickleDB logo"></div> | ||
<div class="body"> | ||
<h1>Welcome</h1> | ||
<strong>pickleDB is a lightweight and simple key-value store.</strong> | ||
It is built upon Python's <a href="https://docs.python.org/3/library/json.html">json</a> | ||
module and was inspired by <a href="http://redis.io/">redis</a>. It is licensed | ||
with the BSD three-clause license. | ||
<h1>pickleDB is Fun</h1> | ||
<code> | ||
<span class="c2">>>></span> <span class="c9">import</span> pickledb | ||
<br /><br /> | ||
<span class="c2">>>></span> db = pickledb.load(<span class="c9">'example.db', False</span>) | ||
<br /><br /> | ||
<div class="logo"> | ||
<img src="logo.png" alt="pickleDB logo"> | ||
</div> | ||
<div class="body"> | ||
<h1>Welcome</h1> | ||
<p><strong>pickleDB is a lightweight, simple and fast key-value store.</strong> | ||
It is built upon the <a href="https://pypi.org/project/orjson/">orjson</a> module for extremely high performance and was inspired by <a href="http://redis.io/">redis</a>. It is licensed under the BSD three-clause license.</p> | ||
|
||
<h2>pickleDB is Fun</h2> | ||
<pre><code> | ||
<span class="c2">>>></span> <span class="c9">from</span> pickledb <span class="c9">import</span> pickleDB | ||
|
||
<span class="c2">>>></span> db = PickleDB(<span class="c9">'example.json'</span>) | ||
|
||
<span class="c2">>>></span> db.set(<span class="c9">'key', 'value'</span>) | ||
<br /> | ||
True | ||
<br /><br /> | ||
|
||
<span class="c2">>>></span> db.get(<span class="c9">'key'</span>) | ||
<br /> | ||
'value' | ||
<br /><br /> | ||
<span class="c2">>>></span> db.dump() | ||
<br /> | ||
|
||
<span class="c2">>>></span> db.save() | ||
True | ||
</code> | ||
<h1>And Easy to Install</h1> | ||
<code> | ||
</code></pre> | ||
|
||
<h2>And Easy to Install</h2> | ||
<pre><code> | ||
<span class="c2">$</span> pip install <span class="c9">pickledb</span> | ||
</code> | ||
<h1>More Information</h1> | ||
You can view all of pickleDB's commands and what they do <a href="commands.html">here</a>.<br /> | ||
<br /> | ||
pickleDB was written by <a href="http://patx.github.io">Harrison Erd</a>. If you would like | ||
to file an issue report or fork the project, | ||
check out the <a href="http://github.com/patx/pickledb">Github project page</a>. | ||
You can also take a look at pickleDB <a href="https://pypi.python.org/pypi/pickleDB">on PyPI</a>.<br /> | ||
<br /> | ||
pickleDB got its name from Python's "pickle" module, which it previously used. | ||
However, now pickleDB uses the "json" module. It is faster and cleaner. | ||
But the name stuck! | ||
<br /> | ||
<br /> | ||
<br> | ||
<a href="http://github.com/patx/pickledb"><img style="position: fixed; top: 0; right: 0; border: 0;" src="http://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" alt="Fork me on GitHub"></a> | ||
</div> | ||
</code></pre> | ||
|
||
<h2>More Information</h2> | ||
<p>You can view all of pickleDB's commands and what they do <a href="commands.html">here</a>.</p> | ||
<p>pickleDB was written by <a href="http://patx.github.io">Harrison Erd</a>. If you would like to file an issue report or fork the project, check out the <a href="https://github.com/patx/pickledb">Github project page</a>. You can also take a look at pickleDB <a href="https://pypi.python.org/pypi/pickleDB">on PyPI</a>.</p> | ||
|
||
<a href="https://github.com/patx/pickledb"> | ||
<img style="position: fixed; top: 0; right: 0; border: 0;" src="https://github.blog/wp-content/uploads/2008/12/forkme_right_green_007200.png" alt="Fork me on GitHub"> | ||
</a> | ||
</div> | ||
</body> | ||
</html> | ||
|
Large diffs are not rendered by default.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,115 +1,126 @@ | ||
|
||
import unittest | ||
import os | ||
import time | ||
from pickledb import load | ||
import signal | ||
from pickledb import PickleDB # Adjust the import path if needed | ||
|
||
class TestPickleDBEnhanced(unittest.TestCase): | ||
|
||
class TestPickleDB(unittest.TestCase): | ||
def setUp(self): | ||
self.db_file = "test_db.json" | ||
self.db = load(self.db_file, auto_dump=True, enable_ttl=True) | ||
"""Set up a PickleDB instance with a real file.""" | ||
self.test_file = "test_pickledb.json" | ||
self.db = PickleDB(self.test_file, auto_dump=False) | ||
|
||
def tearDown(self): | ||
if os.path.exists(self.db_file): | ||
os.remove(self.db_file) | ||
if os.path.exists(f"{self.db_file}.gz"): | ||
os.remove(f"{self.db_file}.gz") | ||
|
||
# Enhanced List Features | ||
def test_lsort(self): | ||
self.db.lcreate("test_list") | ||
self.db.ladd("test_list", 3) | ||
self.db.ladd("test_list", 1) | ||
self.db.ladd("test_list", 2) | ||
self.assertEqual(self.db.lsort("test_list"), [1, 2, 3]) | ||
self.assertEqual(self.db.lsort("test_list", reverse=True), [3, 2, 1]) | ||
|
||
def test_lremove(self): | ||
self.db.lcreate("test_list") | ||
self.db.ladd("test_list", "item1") | ||
self.db.ladd("test_list", "item2") | ||
self.assertTrue(self.db.lremove("test_list", "item1")) | ||
self.assertEqual(self.db.lgetall("test_list"), ["item2"]) | ||
self.assertFalse(self.db.lremove("test_list", "nonexistent")) | ||
|
||
def test_lgetrange(self): | ||
self.db.lcreate("test_list") | ||
self.db.ladd("test_list", "a") | ||
self.db.ladd("test_list", "b") | ||
self.db.ladd("test_list", "c") | ||
self.assertEqual(self.db.lgetrange("test_list", 0, 2), ["a", "b"]) | ||
|
||
def test_llen(self): | ||
self.db.lcreate("test_list") | ||
self.db.ladd("test_list", "a") | ||
self.db.ladd("test_list", "b") | ||
self.assertEqual(self.db.llen("test_list"), 2) | ||
|
||
# Enhanced Dictionary Features | ||
def test_dremove(self): | ||
self.db.dcreate("test_dict") | ||
self.db.dadd("test_dict", "key1", "value1") | ||
self.assertTrue(self.db.dremove("test_dict", "key1")) | ||
self.assertFalse(self.db.dremove("test_dict", "key2")) | ||
|
||
def test_dmerge(self): | ||
self.db.dcreate("test_dict") | ||
self.db.dadd("test_dict", "key1", "value1") | ||
self.db.dmerge("test_dict", {"key2": "value2", "key3": "value3"}) | ||
self.assertEqual(self.db.dgetall("test_dict"), { | ||
"key1": "value1", | ||
"key2": "value2", | ||
"key3": "value3" | ||
}) | ||
|
||
def test_dkeys(self): | ||
self.db.dcreate("test_dict") | ||
self.db.dadd("test_dict", "key1", "value1") | ||
self.db.dadd("test_dict", "key2", "value2") | ||
self.assertEqual(set(self.db.dkeys("test_dict")), {"key1", "key2"}) | ||
|
||
def test_dvalues(self): | ||
self.db.dcreate("test_dict") | ||
self.db.dadd("test_dict", "key1", "value1") | ||
self.db.dadd("test_dict", "key2", "value2") | ||
self.assertEqual(set(self.db.dvalues("test_dict")), {"value1", "value2"}) | ||
|
||
# Additional Tests | ||
def test_persistence(self): | ||
"""Clean up after tests.""" | ||
if os.path.exists(self.test_file): | ||
os.remove(self.test_file) | ||
|
||
def _timeout_handler(self, signum, frame): | ||
"""Handle timeouts for stress tests.""" | ||
raise TimeoutError("Test exceeded the timeout duration") | ||
|
||
# Original Stress Test | ||
def test_stress_operation(self): | ||
"""Stress test: Insert and retrieve a large number of key-value pairs, then dump.""" | ||
timeout_duration = 600 # Timeout in seconds (10 minutes) | ||
|
||
# Set a signal-based timeout | ||
signal.signal(signal.SIGALRM, self._timeout_handler) | ||
signal.alarm(timeout_duration) | ||
|
||
try: | ||
num_docs = 20_000_000 | ||
|
||
# Measure memory loading time | ||
start_time = time.time() | ||
for i in range(num_docs): | ||
self.db.set(f"key{i}", f"value{i}") | ||
mem_time = time.time() | ||
mem_duration = mem_time - start_time | ||
print(f"\n{num_docs} stored in memory in {mem_duration:.2f} seconds") | ||
|
||
# Measure retrieval performance before dumping | ||
start_time = time.time() | ||
retrieved_docs = [self.db.get(f"key{i}") for i in range(num_docs)] | ||
retrieval_time = time.time() - start_time | ||
print(f"Retrieved {num_docs} key-value pairs in {retrieval_time:.2f} seconds") | ||
|
||
# Measure dump performance | ||
start_time = time.time() | ||
self.db.dump() | ||
dump_time = time.time() - start_time | ||
print(f"Dumped {num_docs} key-value pairs to disk in {dump_time:.2f} seconds") | ||
|
||
finally: | ||
signal.alarm(0) # Cancel the alarm after the test | ||
|
||
# Functional Tests | ||
def test_set_and_get(self): | ||
"""Test setting and retrieving a key-value pair.""" | ||
self.db.set("key1", "value1") | ||
del self.db | ||
db = load(self.db_file, auto_dump=True) | ||
self.assertEqual(db.get("key1"), "value1") | ||
|
||
def test_invalid_ladd(self): | ||
with self.assertRaises(TypeError): | ||
self.db.ladd("nonexistent_list", "item") | ||
self.assertEqual(self.db.get("key1"), "value1") | ||
|
||
def test_invalid_dadd(self): | ||
with self.assertRaises(TypeError): | ||
self.db.dadd("nonexistent_dict", "key", "value") | ||
def test_get_nonexistent_key(self): | ||
"""Test retrieving a key that does not exist.""" | ||
self.assertIsNone(self.db.get("nonexistent")) | ||
|
||
def test_compress(self): | ||
def test_remove_key(self): | ||
"""Test removing a key-value pair.""" | ||
self.db.set("key1", "value1") | ||
self.assertTrue(self.db.compress()) | ||
self.assertTrue(os.path.exists(f"{self.db_file}.gz")) | ||
|
||
def test_ttl_expiry(self): | ||
self.db.set("key1", "value1", ttl=1) | ||
time.sleep(2) | ||
self.assertTrue(self.db.remove("key1")) | ||
self.assertIsNone(self.db.get("key1")) | ||
|
||
def test_clear(self): | ||
def test_remove_nonexistent_key(self): | ||
"""Test removing a key that does not exist.""" | ||
self.assertFalse(self.db.remove("nonexistent")) | ||
|
||
def test_purge(self): | ||
"""Test purging all keys and values.""" | ||
self.db.set("key1", "value1") | ||
self.db.set("key2", "value2") | ||
self.db.clear() | ||
self.assertEqual(len(self.db.getall()), 0) | ||
self.db.purge() | ||
self.assertEqual(self.db.all(), []) | ||
|
||
def test_deldb(self): | ||
def test_all_keys(self): | ||
"""Test retrieving all keys.""" | ||
self.db.set("key1", "value1") | ||
self.db.deldb() | ||
self.assertFalse(os.path.exists(self.db_file)) | ||
self.db.set("key2", "value2") | ||
self.assertListEqual(sorted(self.db.all()), ["key1", "key2"]) | ||
|
||
def test_dump_and_reload(self): | ||
"""Test dumping the database to disk and reloading it.""" | ||
self.db.set("key1", "value1") | ||
self.db.dump() | ||
reloaded_db = PickleDB(self.test_file, auto_dump=False) | ||
self.assertEqual(reloaded_db.get("key1"), "value1") | ||
|
||
def test_invalid_file_loading(self): | ||
"""Test initializing a database with a corrupt file.""" | ||
with open(self.test_file, 'w') as f: | ||
f.write("corrupt data") | ||
db = PickleDB(self.test_file, auto_dump=False) | ||
self.assertEqual(db.all(), []) | ||
|
||
def test_auto_dump(self): | ||
"""Test the auto-dump functionality.""" | ||
db = PickleDB(self.test_file, auto_dump=True) | ||
db.set("key1", "value1") | ||
reloaded_db = PickleDB(self.test_file, auto_dump=False) | ||
self.assertEqual(reloaded_db.get("key1"), "value1") | ||
|
||
def test_set_non_string_key(self): | ||
"""Test setting a non-string key.""" | ||
self.db.set(123, "value123") | ||
self.assertEqual(self.db.get("123"), "value123") | ||
|
||
def test_remove_non_string_key(self): | ||
"""Test removing a key that was stored as a non-string key.""" | ||
self.db.set(123, "value123") | ||
self.assertTrue(self.db.remove(123)) | ||
self.assertIsNone(self.db.get("123")) | ||
|
||
|
||
if __name__ == "__main__": | ||
unittest.main() | ||
|