2

I am trying to decide which database I should use for an application I'm making. I was leaning toward dynamoDB because of its scalability, but then I read in the documentation which said:

there is a limit of 64 KB on the item size

although it looks like MySQL has a similar restriction documented here

This application will be storing a lot of data in two arrays, which could contain upwards of 10,000-100,000 strings in each. I estimate that these strings will each be somewhere around 20 characters long, so each element of the array will be around 40bytes and each array could be around 4MB.

Given this predicament, what database on amazon AWS would you use; or how would you get around the limit of size per row?

Community
  • 1
Logan Besecker
  • 145
  • 1
  • 5
  • 2
    Based on the nature of the data wouldn't be better to use an noSQL solution like MongoDB ? – golja Jul 09 '12 at 01:40
  • thanks for pointing out MongoDB, I agree with you about choosing a noSQL database which is why I was originally looking at dynamoDB(the only database amazon supports with their managed databases.) I've been reading some information about MongoDB and it looks like a perfect fit for my application. Thank you very much @golija – Logan Besecker Jul 09 '12 at 02:41
  • Sounds like your model or your selection of a db for storage is wrong. Index of an array should generally equate to rows in a table not columns. – JamesRyan Oct 17 '13 at 10:41

3 Answers3

4

There is a hard limit of 4096 columns per table, but the effective maximum may be less for a given table. The exact limit depends on several interacting factors.

Every table (regardless of storage engine) has a maximum row size of 65,535 bytes. Storage engines may place additional constraints on this limit, reducing the effective maximum row size.

MySql 5.0 Manual

So either DBMS should allow you the same amount of data inside each row. Looks like neither will be sufficient for your needs or for your way to process data.

Anyway, you shouldn't store this much data in each row, you'd probably be better off storing each string separately and using other tables to reference them.

arkihillel
  • 156
  • 2
0

after reading further in the MySQL storage requirement page I noticed this section:

The internal representation of a table has a maximum row size of 65,535 bytes, even if the storage engine is capable of supporting larger rows. This figure excludes BLOB or TEXT columns, which contribute only 9 to 12 bytes toward this size. For BLOB and TEXT data, the information is stored internally in a different area of memory than the row buffer.

Logan Besecker
  • 145
  • 1
  • 5
0

With DynamoDB its recommended you use Amazon S3 for storing larger values.

Other options include gzip compressing the content which will give you about 200KB of storage.

Please see Matt Woods slides below:

http://www.slideshare.net/AmazonWebServices/building-applications-with-dynamodb

Malcolm Edgar
  • 1