Metadata-Version: 2.1
Name: leaky-bucket-py
Version: 0.1.3
Summary: Leaky bucket implementation in python with different persistence options
Author: DuneRaccoon
Author-email: DuneRaccoon <benjamincsherro@hotmail.com>
License: MIT License
        
        Copyright (c) 2024 Benjamin Herro
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Keywords: rate-limiter,asyncio,leaky-bucket,redis,sqlite3
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: redis>=3.5.3

# Py Leaky Bucket

An implementation of the leaky bucket algorithm in python, with different persistence options for use in high throughput, multi process/worker applications. This is a useful package when managing integrations with various API's that have different rate limits.



## What's in it?

The package includes:

- **Leaky Bucket Algorithm**: A flexible rate-limiting algorithm that supports various persistence backends.
- **Persistence Backends**:
  - **In-Memory**: Fast, lightweight, and great for single-process applications.
  - **Redis**: Suitable for distributed, multi-worker environments.
  - **SQLite**: A file-based backend that supports high concurrency in single-node setups (honestly not the best option for high-throughput due to the nature of sqlite and the need to deadlock the database - better to use the redis option if possible)
- **Hourly Limit Support**: Control total operations over a rolling hourly window in addition to per-second rate limits.
- **Thread-Safe and Process-Safe**: Implements proper locking mechanisms to ensure safe concurrent usage.
- **Asynchronous and Synchronous**: Works in both `asyncio`-based and synchronous applications.
- **Decorators and Context Managers**: Simplify integration with your existing functions and methods.



### Installation

Easy to install:

```
pip install leaky-bucket-py
```



## Usage:



#### Redis backend with async bucket:

```python
import asyncio
import redis
from leakybucket.bucket import AsyncLeakyBucket
from leakybucket.persistence.redis import RedisLeakyBucketStorage

# Connect to Redis
redis_conn = redis.Redis(host='localhost', port=6379, db=0)

# Create a new Redis storage backend
storage = RedisLeakyBucketStorage(
    redis_conn,
    redis_key="api_bucket",
    max_rate=5,
    time_period=1
)

# Create a new LeakyBucket instance
bucket = AsyncLeakyBucket(storage)

# Make requests using the bucket as a context manager
async def make_requests():
    async def make_request():
        async with bucket:  # block if the rate limit is exceeded
            print("Making request")
            await asyncio.sleep(1)
    await asyncio.gather(*[make_request() for i in range(10)])


# or use a decorator to rate limit a coroutine
@bucket.throttle()
async def make_request(index):
    print(f"Making request {index}")
    await asyncio.sleep(1)


async def main():
    await make_requests()
    await asyncio.gather(*[make_request(i) for i in range(10)])


asyncio.run(main())
```



#### Memory backend:

###### Synchronous:

```python
import httpx
from leakybucket.bucket import LeakyBucket
from leakybucket.persistence.memory import InMemoryLeakyBucketStorage

# Create a new Memory storage backend (3 requests per second)
storage = InMemoryLeakyBucketStorage(max_rate=3, time_period=1)

# Create a new LeakyBucket instance
throttler = LeakyBucket(storage)

@throttler.throttle()
def fetch_data(api_url: str):
    response = httpx.get(api_url)
    data = response.json()
    print(data)
    return data

def main():
    # make multiple requests
    api_url = "https://jsonplaceholder.typicode.com/posts/1"
    results = []
    for _ in range(10):
        results.append(fetch_data(api_url))
    print(results)

main()

```

###### Asynchronous:

```python
import asyncio
import httpx
from leakybucket.bucket import AsyncLeakyBucket
from leakybucket.persistence.memory import InMemoryLeakyBucketStorage

# Create a new Memory storage backend (3 requests per second)
storage = InMemoryLeakyBucketStorage(max_rate=3, time_period=1)

# Create a new LeakyBucket instance
async_throttler = AsyncLeakyBucket(storage)

@async_throttler.throttle()
async def async_fetch_data(api_url):
    async with httpx.AsyncClient() as client:
        response = await client.get(api_url)
        data = response.json()
        print(data)
        return data

async def main():
    # make multiple requests
    api_url = "https://jsonplaceholder.typicode.com/posts/1"
    tasks = [async_fetch_data(api_url) for _ in range(10)]
    results = await asyncio.gather(*tasks)
    print(results)

asyncio.run(main())

```



#### Sqlite backend:

```python
import time
from leakybucket.bucket import LeakyBucket
from leakybucket.persistence.sqlite import SqliteLeakyBucketStorage

# Create a shared SQLite bucket
bucket = LeakyBucket(
    SqliteLeakyBucketStorage(
        db_path="leakybucket.db", 
        max_rate=10, 
        time_period=10
    )
)

# Decorate the function
@bucket.throttle()
def make_request(index):
    print(f"Making request {index}")

def main():
    for i in range(35):
        make_request(i)

main()

```




