python - How To Store Changing Data For 20,000 Items In A Database -

python - How To Store Changing Data For 20,000 Items In A Database -

i have 20,000 different items i'd track. have same attributes: name, height, weight, assigned id, , cost (these examples). attributes, name , assigned id, won't alter ever. other attributes alter rapidly (5 - 10 times hour) , (1 - 2 times week or month) others. way track these changes in database can perform analysis on changes.

now, have no experience databases. have never dealt amount of info before , hence have gotten csv files. also, using python code retrieve info , using analyse well. hence, thinking sqlite, believe pretty easy utilize python, might best option. thinking create table each item , have attributes columns , changes on each row. however, feeling there much improve (efficiency , storage size wise) ways this.

if please outline best way store info regular retrieval analysis wouldn't difficult, appreciated.

also, if point me resources doing info analysis on type of info in python great too.

thanks in advance

for simple database stuff, sqlalchemy friend: http://www.sqlalchemy.org/

the documentation includes a comprehensive tutorial goes through high-level concepts involved in working database, how design , work tables straight in python.

here's sample, showing how can define tables in code. sqlalchemy take care of managing database, , can work them normal python objects.

>>> sqlalchemy import column, integer, string >>> class user(base): ... __tablename__ = 'users' ... ... id = column(integer, primary_key=true) ... name = column(string) ... fullname = column(string) ... password = column(string) ... ... def __repr__(self): ... homecoming "<user(name='%s', fullname='%s', password='%s')>" % ( ... self.name, self.fullname, self.password)

plus, tutorial uses sqlite database!

i'd recommend taking around database you're working on it, help improve sense of what's going on under hood. didn't mention environment you're working in, there lot of different options exploring sqlite databases: sqlite3 in linux, sqlite database browser in windows, etc.

update:

as other part of question, might want read on database normalization.

the general thought of (basic) normalization different pieces of similar info should kept in 1 place, rather beingness repeating in lots of different tables.

from brief description you've given, approach widgets table store id , name each item you're tracking, , dimensions table list changes:

+-----------+----------------------+--------+--------+-----+ | widget_id | change_time | height | weight | ... | +-----------+----------------------+--------+--------+-----+ | 0 | 05-12-2014 18:30:58 | 4 | 10 | ... | | 0 | 06-22-2014 12:32:46 | 4 | 12 | ... | | 0 | 06-23-2014 01:02:00 | 6 | 12 | ... | | ... |

whenever alter occurs, add together entry dimensions table. way, can dimensions time in past, analysis of changes on time, etc.

python database-design data-analysis