从DynamoDb查询的Python脚本未提供所有项目

问题描述

我已经编写了以下python代码以从表中获取数据，但未按我的要求获取所有项目。当我检查DynamoDb的AWS控制台页面时，与从脚本中获取的内容相比，我看到的条目更多。

I have written following python code to fetch data from a table but its not fetching all the items as I want. When I check on AWS console page of DynamoDb, I can see much more entries as compared to what I get from script.

from __future__ import print_function # Python 2/3 compatibility
import boto3
import json
import decimal
from datetime import datetime
from boto3.dynamodb.conditions import Key, Attr
import sys

# Helper class to convert a DynamoDB item to JSON.
class DecimalEncoder(json.JSONEncoder):
    def default(self, o):
        if isinstance(o, decimal.Decimal):
            if o % 1 > 0:
                return float(o)
            else:
                return int(o)
        return super(DecimalEncoder, self).default(o)

dynamodb = boto3.resource('dynamodb', aws_access_key_id = '',
        aws_secret_access_key = '',
        region_name='eu-west-1', endpoint_url="http://dynamodb.eu-west-1.amazonaws.com")

mplaceId = int(sys.argv[1])
table = dynamodb.Table('XYZ')

response = table.query(
    KeyConditionExpression=Key('mplaceId').eq(mplaceId)
)

print('Number of entries found ', len(response['Items']))

我也从aws控制台执行了同样的操作。通过mplaceId查询。

I did the same thing from aws console also. Query by mplaceId.

发生这种情况的任何原因吗？

Any reason why its happening?

推荐答案

dynamodb.Table.query（）返回最大1MB的数据。从：

dynamodb.Table.query() returns at max 1MB of data. From the boto3 documentation:

实际上并没有 boto3 的限制，但对基础查询 -API有所限制。

That's actually no boto3-limitation, but a limitation of the underlying query-API.

除了自己实施分页功能外，您还可以使用 boto3 的。这是一个显示：

Instead of implementing pagination yourself, you can use boto3's built-in pagination . Here is an example showing the use of the paginator for querying DynamoDB tables provided by boto3:

dynamodb_client = boto3.client('dynamodb')
paginator = dynamodb_client.get_paginator('query')
page_iterator = paginator.paginate(KeyConditionExpression=Key('mplaceId').eq(mplaceId))

for page in page_iterator:
    print(page['Items'])

这篇关于从DynamoDb查询的Python脚本未提供所有项目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！