Initial docs for ft_utils

Summary: An initial set of docs (less the worked example) for ft_utils including some guidance on the programming style recommended for FTPython when using this library.

Reviewed By: alexmalyshev

Differential Revision: D62376690

fbshipit-source-id: 8e17edf67a4f5a15f38b7671bcdf407c795a9c4f

Author: SonicField

README.md

@@ -23,6 +23,10 @@ In addition to these guidelines, we have a few extra rules to keep in mind:
 * **Unit Tests**: Every feature must have at least one unit test to ensure it works correctly.
 * **Benchmarks**: It is best practice to write benchmarks for all new features to further test and to drive our aim of high performance.
 
+## Documentation
+
+See to [documentation](docs/introduction.md).
+
 ## Build And CI
 
 ### Source Only

docs/atomicity_in_Python.md

@@ -0,0 +1,68 @@
+# Solving Atomicity/Consistency Issues in Free Threaded Python
+
+## Introduction
+
+### Atomicity/Consistency
+
+These are similar concepts in multi-threaded programming. A change to data (setting a variable, updating a dictionary etc.) is atomic if it is either completely done or not done at all. Consistency here is really thread-consistency where all threads in a program see exactly the same data at the same time.
+
+In traditional Python, the Global Interpreter Lock (GIL) ensures that memory operations are implicitly atomic and consistent. For example, if we update a variable from 1 to 2 it is either done or not done and all threads will see the update.
+
+```python
+x = 1
+x = 2
+# x will be 2 in all threads
+
+y = {"a": "b"}
+y["a"] = "c"
+# y["a"] will be "c" in all threads everywhere.
+```
+
+However, with the introduction of Free Threaded Python (PEP 703), we no longer have the GIL to enforce atomicity so changes have been made to the way key parts of the Python runtime work to keep the consistent, atomic behavior.
+
+## Atomic Variables and Data Structures
+
+Most inter-thread communication in Python occurs through variables, which are either slots or dictionary key-value pairs under the covers. To keep Python behaving as expected, these variables have been made atomic. Additionally, data structures like lists are atomic for read/write operations that do not change their size.
+
+## Critical Sections
+
+To prevent native code from experiencing race conditions, Critical Sections are used to protect sensitive areas of code. The native code module of objects has been enhanced to all objects have a 'monitor' which controls its state of locking for that object. A Critical Section takes a lock based on an object's monitor, ensuring that only one thread can execute the protected code at a time. This mechanism provides atomicity and consistency where required, without exposing any locking mechanisms to Python programmers. To the programmer everything works just like it always did except more than one thread can run at once outside of the critical section.
+
+## ft_utils Library
+
+The ft_utils library provides specialized structures and utilities to support fast and scalable free threaded algorithms. Some of the key features of ft_utils include:
+
+*   **ConcurrentDict:** A thread-safe dictionary implementation that scales well with the number of threads accessing it.
+*   **AtomicInt64:** A true atomic integer that does not require locks and supports all pure integer operations with atomic updates.
+*   **BatchExecutor:** A utility that executes tasks in batches, allowing for efficient and thread-safe execution of code that cannot be made free threaded.
+*   **LocalWrapper:** A wrapper that helps manage reference counts between threads by creating a local copy of an object.
+*   **IntervalLock:** A lock that mimics the behavior of the GIL, allowing a program to schedule execution of different threads.
+*   **RWLock:** A readers/writer lock that allows efficient management of a resource read frequently by many threads but updated infrequently.
+*   **ConcurrentGatheringIterator:** An iterator that allows sequenced collection of objects from many threads and reading from one.
+*   **ConcurrentQueue:** A highly scalable queue that does not require locks around push or pop operations.
+*   **AtomicFlag:** A boolean settable flag based on AtomicInt64.
+*   **AtomicReference:** A reference that can be set, get, exchanged, and compared atomically without requiring locks.
+
+## Atomicity in Python
+
+Python is a language which is implemented on a runtime. Python as a language does not say much about atomicity consistency; however, CPython, which is the reference implementation, does have strong guarantees (see the introduction at the start of this document). In this section we will dig into the subject a little more.
+
+While the removal of the GIL in Free Threaded Python introduces new challenges for ensuring atomicity, it's essential to recognize that many Python programming constructs are not inherently atomic or thread-safe.
+
+For instance, even simple operations like `x += 1` can be problematic when dealing with immutable objects like integers. In this case, the operation is not performed in-place; instead, a new value is allocated to `x`, which can lead to unexpected behavior in multi-threaded environments.
+
+In traditional GIL-based Python, some code may appear to work correctly due to the GIL's scheduling mechanism, which prevents threads from running concurrently. However, this does not necessarily mean that the code is thread-safe or atomic.
+
+The GIL itself is not a traditional lock; it uses a combination of opcode counting and explicit release mechanisms to switch between threads. This approach results in locks being acquired and released relatively infrequently in GIL-based Python.
+
+In contrast, Free Threaded Python allows locks to be acquired and released much more rapidly, which can introduce performance issues related to operating system overhead rather than contention. To mitigate these concerns, developers should focus on using high-level concurrency abstractions and data structures designed specifically for parallel execution.
+
+One effective way to achieve robust and scalable concurrent programming in Free Threaded Python is by leveraging Thread Pool Executors and concurrent data types such as ConcurrentDict, AtomicInteger, AtomicReference, and others. By combining these tools with well-established programming patterns, developers can create efficient, reliable, and maintainable concurrent code.
+
+## Conclusion
+
+Solving atomicity issues in Free Threaded Python requires a combination of atomic variables and data structures, Critical Sections to protect native code, and specialized libraries like ft_utils. By using these tools and techniques, developers can write efficient and thread-safe code that takes advantage of the benefits of Free Threaded Python.
+
+## Next Steps
+
+In the next section, [FTPython Programming A Worked Example](ft_worked_example.md), we will demonstrate how to apply these concepts in real-world code, showcasing the benefits of using Free Threaded Python and its associated libraries for concurrent programming.

docs/concurrent_api.md

@@ -0,0 +1,245 @@
+# concurrent Module Documentation
+
+The concurrent module provides a set of concurrently scalable data structures and patterns for Python. This module is designed to support high-performance, scalable programming with Free Threaded Python.
+
+## ConcurrentDict
+
+A concurrently accessible dictionary.
+
+### Methods
+
+* `__init__(scaling=17)`: Initializes a new ConcurrentDict with the specified number of concurrent structures. This relates to the number of threads it supports with good scaling. For optimal performance, this value should be close to the number of cores on the machine. However, under or over estimating this value by a factor of 2 or even more does not have a huge impact on performance.
+* `get(key)`: Returns the value associated with the specified key.
+* `set(key, value)`: Sets the value associated with the specified key.
+* `has(key)`: Returns True if the key is present in the dictionary, False otherwise.
+
+### Operators
+
+ConcurrentDict also supports the following access methods:
+
+* `d[key]`: Returns the value associated with the specified key.
+* `d[key] = value`: Sets the value associated with the specified key.
+* `del d[key]`: Deletes the key-value pair associated with the specified key.
+
+Note that the `in` operator is not supported because of implementation details in the CPython runtime. Please use `has` instead.
+
+### Notes
+
+ConcurrentDict does not support all the API methods of a built-in dict. It is designed for basic key-value store operations in a concurrent environment.
+
+### Example
+```python
+from ft_utils.concurrent import ConcurrentDict
+
+d = ConcurrentDict()
+d['key'] = 'value'
+print(d['key'])  # prints 'value'
+print(d.has('key'))  # prints True
+del d['key']
+print(d.has('key'))  # prints False
+```
+
+## AtomicInt64
+
+A 64-bit integer that can be updated atomically.
+
+### Methods
+
+* `__init__(value=0)`: Initializes a new AtomicInt64 with the specified value.
+* `get()`: Returns the current value.
+* `set(value)`: Sets the value.
+* `incr()`: Increments the value and returns the new value.
+* `decr()`: Decrements the value and returns the new value.
+
+In addition the following numeric methods are implemented.
+
+* `__sub__(self, other)`: Returns the result of subtracting `other` from `self`.
+* `__mul__(self, other)`: Returns the result of multiplying `self` by `other`.
+* `__floordiv__(self, other)`: Returns the largest whole number less than or equal to the result of dividing `self` by `other`.
+* `__or__(self, other)`: Returns the bitwise OR of `self` and `other`.
+* `__xor__(self, other)`: Returns the bitwise XOR of `self` and `other`.
+* `__and__(self, other)`: Returns the bitwise AND of `self` and `other`.
+* `__neg__(self)`: Returns the negative of `self`.
+* `__pos__(self)`: Returns `self` unchanged.
+* `__abs__(self)`: Returns the absolute value of `self`.
+* `__invert__(self)`: Returns the bitwise NOT of `self`.
+* `__bool__(self)`: Returns `True` if `self` is non-zero, `False` otherwise.
+* `__iadd__(self, other)`: Adds `other` to `self` in-place and returns `self`.
+* `__isub__(self, other)`: Subtracts `other` from `self` in-place and returns `self`.
+* `__imul__(self, other)`: Multiplies `self` by `other` in-place and returns `self`.
+* `__ifloordiv__(self, other)`: Divides `self` by `other` in-place using floor division and returns `self`.
+* `__ior__(self, other)`: Performs a bitwise OR of `self` and `other` in-place and returns `self`.
+* `__ixor__(self, other)`: Performs a bitwise XOR of `self` and `other` in-place and returns `self`.
+* `__iand__(self, other)`: Performs a bitwise AND of `self` and `other` in-place and returns `self`.
+* `__int__(self)`: Returns an integer representation of `self`.
+
+### Example
+
+```python
+from ft_utils.concurrent import AtomicInt64
+
+i = AtomicInt64(10)
+print(i.get())  # prints 10
+i.incr()
+print(i.get())  # prints 11
+i.add(5)
+print(i.get())  # prints 16
+```
+
+## AtomicReference
+
+A reference that can be updated atomically.
+
+### Methods
+
+* `__init__(obj=None)`: Initializes a new AtomicReference with the specified object.
+* `get()`: Returns the current object.
+* `set(obj)`: Sets the object.
+* `exchange(obj)`: Exchanges the current object with the specified object and returns the previous object.
+* `compare_exchange(expected, obj)`: Compares the current object with the expected object and exchanges it with the specified object if they match. Returns `True` if the exchange happened, `False` otherwise.
+
+### Using compare_exchange
+
+The `compare_exchange` method can be used in a loop to atomically update the reference, similar to using the CAS instruction in native programming.
+
+### Example
+
+```python
+from ft_utils.concurrent import AtomicReference
+
+r = AtomicReference(0)
+
+def increment(r):
+    while True:
+        current = r.get()
+        new_value = current + 1
+        if r.compare_exchange(current, new_value):
+            break
+
+increment(r)
+print(r.get())  # prints 1
+```
+
+In this example, the `increment` function uses a loop to atomically increment the value of the AtomicReference. The `compare_exchange` method is used to check if the current value is still the same as the expected value, and if so, updates the value to the new value. If another thread has updated the value in the meantime, the `compare_exchange` method will return `False` and the loop will retry.
+
+Here are the documents for the new classes:
+
+## AtomicFlag
+
+A boolean flag that can be updated atomically.
+
+### Methods
+
+* `__init__(value)`: Initializes a new AtomicFlag with the specified value.
+* `set(value)`: Sets the value of the flag.
+* `__bool__()`: Returns the current value of the flag.
+
+### Example
+```python
+from ft_utils.concurrent import AtomicFlag
+
+flag = AtomicFlag(True)
+print(flag)  # prints True
+flag.set(False)
+print(flag)  # prints False
+```
+
+## ConcurrentGatheringIterator
+
+A concurrent iterator that gathers values from multiple threads and yields them in order.
+
+### Methods
+
+* `__init__(scaling)`: Initializes a new ConcurrentGatheringIterator with the specified scaling factor.
+* `insert(key, value)`: Inserts a key-value pair into the iterator.
+* `iterator(max_key, clear)`: Returns an iterator that reads and deletes key-value pairs from the iterator in order.
+
+### Notes
+
+* The iterator uses a ConcurrentDict to store the key-value pairs.
+* The `insert` method is thread-safe and can be called from multiple threads.
+* The `iterator` method returns an iterator that yields the values in order, blocking if the next value is not available.
+* max_key passed to iterator tells the iterator at what point to stop iteration; i.e. all expected values have been gathered..
+* If an exception occurs during insertion, the iterator will fail with a RuntimeError.
+* scaling passed to the __init__ function governs the number of threads the iterator supports with good scaling.
+
+### Example
+
+```python
+from ft_utils.concurrent import ConcurrentGatheringIterator
+
+iterator = ConcurrentGatheringIterator()
+iterator.insert(0, 'value0')
+iterator.insert(1, 'value1')
+iterator.insert(2, 'value2')
+
+for value in iterator.iterator(2):
+    print(value)  # prints 'value0', 'value1', 'value2'
+```
+
+A more complex example:
+
+```python
+from ft_utils.concurrent import ConcurrentGatheringIterator, AtomicInt64
+from concurrent.futures import ThreadPoolExecutor
+
+def insert_value(iterator, atomic_index, value):
+    index = atomic_index.incr()
+    iterator.insert(index, value)
+
+def test_concurrent_gathering_iterator():
+    iterator = ConcurrentGatheringIterator()
+    atomic_index = AtomicInt64(-1)
+
+    with ThreadPoolExecutor(max_workers=10) as executor:
+        futures = []
+        for i in range(100):
+            futures.append(executor.submit(insert_value, iterator, atomic_index, i))
+
+        for future in futures:
+            future.result()
+
+    results = list(iterator.iterator(99))
+    assert results == list(range(100))
+
+test_concurrent_gathering_iterator()
+```
+
+In this example, we use a ThreadPoolExecutor to insert values into the ConcurrentGatheringIterator from multiple threads. We use an AtomicInt64 to generate the indices in order. After inserting all the values, we retrieve the results from the iterator and check that they are in the correct order.
+
+Note that the `insert_value` function is a helper function that inserts a value into the iterator at the next available index. The `test_concurrent_gathering_iterator` function is the main test function that creates the iterator, inserts values from multiple threads, and checks the results.
+
+This example demonstrates that the ConcurrentGatheringIterator can handle concurrent inserts from multiple threads and still produce the correct results in order.
+
+## ConcurrentQueue
+
+A concurrent queue that allows multiple threads to push and pop values.
+
+### Methods
+
+* `__init__(scaling)`: Initializes a new ConcurrentQueue with the specified scaling factor.
+* `push(value)`: Pushes a value onto the queue.
+* `pop()`: Pops a value from the queue.
+
+### Notes
+
+* The queue uses a ConcurrentDict to store the values.
+* The `push` method is thread-safe and can be called from multiple threads.
+* The `pop` method returns the next value in the queue, blocking if the queue is empty.
+* If an exception occurs during push, the queue will fail with a RuntimeError.
+* scaling passed to the __init__ function governs the relates to the number of threads it supports with good scaling.
+
+### Example
+
+```python
+from ft_utils.concurrent import ConcurrentQueue
+
+queue = ConcurrentQueue()
+queue.push('value0')
+queue.push('value1')
+queue.push('value2')
+
+print(queue.pop())  # prints 'value0'
+print(queue.pop())  # prints 'value1'
+print(queue.pop())  # prints 'value2'
+```

docs/ft_utils_api.md

@@ -0,0 +1,18 @@
+# ft_utils API Documentation
+
+## Modules
+
+The ft_utils API is organized into several modules, each providing a specific set of functionalities.
+
+### concurrent
+
+The [concurrent module](concurrent_api.md) provides a set of foundational structures to implement scalable and efficient Free Threaded code. Using these structures can help avoid bottlenecks and enhance thread-consistency in FTPython code.
+
+### synchronization
+
+The [synchronization module](synchronization_api.md) provides specialized lock types that work in GIL-based Python and offer significant benefits for Free Threaded programming. These locks can help improve the performance and reliability of concurrent code.
+
+### local
+
+The [local module](local_api.md) provides helper classes to move certain forms of processing from being cross-thread to being local to a thread. These classes are necessary to avoid key bottlenecks that can occur due to cache contention, critical sections, and reference counting in Free Threaded Python.
+

docs/ft_worked_example.md

@@ -0,0 +1 @@
+** To Be Done **

docs/introduction.md

@@ -0,0 +1,21 @@
+# Introduction to Free Threaded Python and ft_utils
+
+Welcome to the documentation for ft_utils, a library designed to support fast and scalable concurrent programming in Free Threaded Python. This documentation aims to provide not only a comprehensive guide to the ft_utils API but also an introduction to the concepts and best practices of concurrent programming in Free Threaded Python.
+
+## What is Free Threaded Python?
+
+Free Threaded Python is a build of the Python interpreter that removes the Global Interpreter Lock (GIL), allowing true parallel execution of threads. This change enables developers to take full advantage of multi-core processors and write high-performance concurrent code. However, it also introduces new challenges, such as ensuring atomicity and thread safety.
+
+To help you navigate these challenges, we've included a section on [Atomicity in Python](atomicity_in_Python.md), which discusses the implications of removing the GIL and provides guidance on how to ensure thread safety in your code.
+
+## ft_utils Library
+
+The ft_utils library provides a set of tools and data structures designed to make concurrent programming in Free Threaded Python easier and more efficient. The library includes features such as atomic integers, concurrent dictionaries, and batch executors, all of which are designed to help you write fast and scalable concurrent code.
+
+For a comprehensive overview of the ft_utils API, please see our [API documentation](ft_utils_api.md).
+
+## Getting Started
+
+If you're new to concurrent programming in Free Threaded Python, we recommend starting with our [worked example](ft_worked_example.md), which demonstrates how to use ft_utils to write a simple concurrent program. This example will give you a hands-on understanding of how to apply the concepts and best practices discussed in this documentation.
+
+We hope this documentation helps you get started with concurrent programming in Free Threaded Python and makes the most of the ft_utils library. If you have any questions or feedback, please don't hesitate to reach out.