python - Referencing Rows in Instance Variable -
apologies if answered elsewhere, cannot find (or maybe understand) appears trying do...
i have large-ish file approx. 60000 rows , 136 columns. want extract 20 columns , have created instance variable store data. data gets parsed , cut according entries in several of columns , moved new copies of instance variable maintain consistency. @ moment doing each column in instance , copy whole row @ time, cannot find way of referencing row in instance variable. examples of doing , below.
if either point me in right direction or suggest better method of doing same thing (instead of using instance variable), grateful.
instance variable:
class instance_object: ###------------------------------------------------------------------ ### __init__ function create object ### ### stores interesting parameters data ###------------------------------------------------------------------ def __init__(self): self.a = [] self.b = [] self.c = [] # etc
current main code:
result = pyf.open(datapath+data_filename) header = result[1].header file_data = result[1].data data_a, data_b = instance_object(), instance_object() data_a.a = file_data.var1 data_a.b = file_data.var2 data_a.c = file_data.var3 #(etc) in range(len(data_a.a)): if 0.0 <= data_a.a[i] <= 100.0: if -1.0 <= data_a.b[i] <= 1.0: data_b.a.append(data_a.a[i]) data_b.b.append(data_a.b[i]) data_b.c.append(data_a.c[i]) # etc
what do:
for in range(len(data_a.a)): if 0.0 <= data_a.a[i] <= 100.0: if -1.0 <= data_a.b[i] <= 1.0: data_b.append(data_a[i]) # i.e. append entire row data_a data_b whilst # maintaining structure
instead of storing columns of data, store list of rows.
instead of storing columns, picking desired columns, filtering desired rows, can combined operations:
def file_to_rows(fname, line_to_row=none, row_filter=none, skip_header=false, mode="r"): line_to_row |= lambda line: line row_filter |= lambda row: true open(fname, mode) inf: line in inf: row = line_to_row(line) if row_filter(row): yield row def line_to_row_items(items): max_split = max(items) + 1 def fn(line): row = line.split(none, max_split) return [row[item] item in items] return fn def row_filter(row): return ( 0. <= row[0] <= 100. , -1. <= row[1] <= 1. ) data = list(file_to_rows( "mydata.txt", line_to_row_items([0, 1, 4, 5, 6, 9, 10, 15, 18, 21]), # columns keep skip_header = true ))
Comments
Post a Comment