Skip to content

Commit 5270f51

Browse files
committed
Added async throttling section to readme.
Updated changelog for v5.
1 parent 39d8e32 commit 5270f51

File tree

3 files changed

+33
-4
lines changed

3 files changed

+33
-4
lines changed

CHANGELOG.md

+17
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,22 @@
11
# Porter change log
22

3+
## 5.0.0 – Async
4+
5+
Porter v5 introduces asynchronous imports and complete strict type safety (excluding union types and generics).
6+
7+
### Breaking changes
8+
9+
* Removed support for PHP 5.5, 5.6 and 7.0.
10+
* Every interface has been updated to include return types which means all consuming projects must also add the same return type.
11+
* Replaced `Connector::fetch` string source parameter with new `DataSource` interface.
12+
* Removed `ConnectionContext` from `Connector` interface.
13+
* Added `SingleRecordResource` interface that resources must implement to be used with `Porter::importOne()`.
14+
* Prevented single record resources being imported with multi-record import methods.
15+
* Replaced `RecoverableConnectorException` with `RecoverableException` interface.
16+
* Removed failed abstractions: `ConnectorOptions` and `EncapsulatedOptions`.
17+
* Removed abstraction: `CacheKeyGenerator`.
18+
* Moved `ForeignResourceException` to Porter's namespace.
19+
320
## 4.0.0 – Rewrite
421

522
Porter v4 fixes all known design flaws (#31, #43) and critically re-evaluates every part of Porter's design. All base classes have been discarded (`AbstractProvider`, `AbstractResource`), moving their code within Porter, relying solely on interfaces instead. This frees up the inheritance chain for applications to use as they wish, making it much easier to integrate Porter into existing projects.

README.md

+15-3
Original file line numberDiff line numberDiff line change
@@ -189,8 +189,18 @@ We must be inside the async event loop to begin programming asynchronously. Let'
189189
});
190190
```
191191

192+
We would not usually code directly inside the event loop in a real application, however we always need to create the event loop somewhere, even if it just calls a service method in our application which delegates to other objects. To pass asynchronous data through layers of abstraction, our application's methods must return `Promise`s that wrap the data they would normally return directly in a synchronous application. For example, a method returning `string` would instead return `Promise<string>`, that is, a promise that returns a string.
193+
192194
Programming asynchronously requires an understanding of Amp, the async framework. Further details can be found in the official [Amp documentation][].
193195

196+
### Throttling
197+
198+
The asynchronous import model is very powerful because it changes our application's performance from being I/O-bound to being CPU-bound. That is, in the traditional synchronous model, each import operation must wait for the previous to complete before the next begins, meaning the total import time depends how long it takes each import's network I/O to complete. In the async model, since we send many requests concurrently without waiting for the previous to complete, on average each import operation will only take as long as our CPU takes to process it, since we are busy processing another import during network latency.
199+
200+
High volume synchronous imports are, in a way, self-throttling and it is rare to trip protection measures in this mode, however the naïve approach to asynchronous imports is often fraught with perils. For example, when we import 10,000 HTTP resources at once, one of two things usually happens: either we run out of PHP memory and the process is killed or the HTTP server blocks us for sending too many requests in a short period. The solution is throttling.
201+
202+
We provide [Async Throttle][] to throttle asynchronous imports. The Async Throttle is a separate project that does not have any direct integration with Porter because that is not needed. The throttle operates on any Amp promises, such as those returned by Porter. The throttle works by preventing additional operations starting when too many are concurrently executing, based on user-defined limits.
203+
194204
Transformers
195205
------------
196206

@@ -468,9 +478,10 @@ Limitations
468478

469479
Current limitations that may affect some users and should be addressed in the near future.
470480

471-
- No end-to-end data steaming interface yet.
472-
- Caching does not support asynchronous imports yet.
473-
- [Sub-imports][] do not support async yet.
481+
- No end-to-end data steaming interface.
482+
- Caching does not support asynchronous imports.
483+
- [Sub-imports][] do not support async.
484+
- No import rate throttle for synchronous imports.
474485

475486
Testing
476487
-------
@@ -533,3 +544,4 @@ Porter is published under the open source GNU Lesser General Public License v3.0
533544
[ECB test]: https://github.com/Provider/European-Central-Bank/blob/master/test/DailyForexRatesTest.php
534545
[Amp]: https://amphp.org
535546
[Amp documentation]: https://amphp.org/amp/
547+
[Async Throttle]: https://github.com/ScriptFUSION/Async-Throttle

composer.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
},
2626
"suggest" : {
2727
"connectors/http": "Provides an HTTP connector for Porter providers.",
28-
"transformers/mapping-transformer": "Transforms records using Mappings.",
28+
"transformers/mapping-transformer": "Transforms records using Mappings and provides sub-imports.",
2929
"async/throttle": "Limits throughput of asynchronous imports."
3030
},
3131
"autoload": {

0 commit comments

Comments
 (0)