python - Complete scan of dynamoDb with boto3 -

my table around 220mb 250k records within it. i'm trying pull of data python. realize needs chunked batch process , looped through, i'm not sure how can set batches start previous left off.

is there way filter scan? read filtering occurs after loading , loading stops @ 1mb wouldn't able scan in new objects.

any assistance appreciated.

import boto3 dynamodb = boto3.resource('dynamodb',     aws_session_token = aws_session_token,     aws_access_key_id = aws_access_key_id,     aws_secret_access_key = aws_secret_access_key,     region_name = region     )  table = dynamodb.table('widgetstablename')  data = table.scan() 

boto3 offers paginators handle pagination details you. here doc page scan paginator. basically, use so:

import boto3  client = boto3.client('dynamodb') paginator = client.get_paginator('scan')  page in paginator.paginate():     #