CMS 3D CMS Logo
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Properties Friends Macros Groups Pages
Functions
data_formats Namespace Reference

Functions

def _dicts_to_orm_objects
 
def _objects_to_dicts
 
def _to_array_of_dicts
 
def _to_datatables
 
def apply_function
 
def objects_to_dicts
 
def query
 
def to_array_of_dicts
 
def to_datatables
 
def to_sql_query
 

Detailed Description

This file holds decorator functions that can rearrange data returned from data sources.
They should be used to decorate the method that holds the script that is being passed to the framework.

Note: may also contain a decorator that can wrap a class around a function that contains a script (future development).

Function Documentation

def data_formats._dicts_to_orm_objects (   model,
  data 
)
private

Definition at line 102 of file data_formats.py.

References isotrackApplyRegressor.model, and sistrip::SpyUtilities.range().

Referenced by models.generate(), and data_formats_tests.data_formats_tests.test_dicts_to_orm_objects().

102 
103 def _dicts_to_orm_objects(model, data):
104  if data.__class__.__name__ in ["json_list", "json_dict", "json_basic"]:
105  data = data.data()
106  new_data = [model(data[n]) for n in range(0, len(data))]
return new_data
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
data_formats._dicts_to_orm_objects
def _dicts_to_orm_objects
Definition: data_formats.py:102
def data_formats._objects_to_dicts (   data)
private

Definition at line 96 of file data_formats.py.

References sistrip::SpyUtilities.range().

Referenced by data_sources.json_list.as_table(), objects_to_dicts(), data_formats_tests.data_formats_tests.test_dicts_to_orm_objects(), and data_formats_tests.data_formats_tests.test_orm_objects_to_dicts().

96 
97 def _objects_to_dicts(data):
98  if data.__class__.__name__ in ["json_list", "json_dict", "json_basic"]:
99  data = data.data()
100  new_data = [data[n].as_dicts() for n in range(0, len(data))]
101  return json_data_node.make(new_data)
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
data_formats._objects_to_dicts
def _objects_to_dicts
Definition: data_formats.py:96
def data_formats._to_array_of_dicts (   data)
private

Definition at line 59 of file data_formats.py.

References data, str, and ComparisonHelper.zip().

Referenced by to_array_of_dicts().

59 
60 def _to_array_of_dicts(data):
61  # check to see if the user has returned a data source, instead of a json data node
62  if not(data.__class__.__name__ in ["json_list", "json_dict", "json_basic"]):
63  data = json_data_node.make(data)
64  headers = data.get("headers").data()
65  data_list = data.get("data").data()
66  def unicode_to_str(string):
67  return str(string) if type(string) == str else string
68  headers = list(map(unicode_to_str, headers))
69  def row_to_dict(row):
70  row = list(map(unicode_to_str, row))
71  return dict(list(zip(headers, row)))
72  array_of_dicts = list(map(row_to_dict, data_list))
73  return json_data_node.make(array_of_dicts)
ComparisonHelper::zip
OutputIterator zip(InputIterator1 first1, InputIterator1 last1, InputIterator2 first2, InputIterator2 last2, OutputIterator result, Compare comp)
Definition: L1TStage2CaloLayer1.h:41
data
char data[epos_bytes_allocation]
Definition: EPOS_Wrapper.h:79
data_formats._to_array_of_dicts
def _to_array_of_dicts
Definition: data_formats.py:59
str
#define str(s)
Definition: TestProcessor.cc:53
def data_formats._to_datatables (   data)
private

Definition at line 74 of file data_formats.py.

References data, relativeConstraints.keys, sistrip::SpyUtilities.range(), str, and makeHLTPrescaleTable.values.

74 
75 def _to_datatables(data):
76  headers = list(map(str, list(data.get(0).data().keys())))
77  new_data = []
78  for n in range(0, len(data.data())):
79  new_data.append([str(entry) if type(entry) == str else entry for entry in list(data.get(n).data().values())])
80  return json_data_node.make({
81  "headers" : headers,
82  "data" : new_data
83  })
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
data_formats._to_datatables
def _to_datatables
Definition: data_formats.py:74
data
char data[epos_bytes_allocation]
Definition: EPOS_Wrapper.h:79
str
#define str(s)
Definition: TestProcessor.cc:53
def data_formats.apply_function (   data,
  function,
  key 
)

Definition at line 88 of file data_formats.py.

References callgraph.function, and sistrip::SpyUtilities.range().

88 
89 def apply_function(data, function, key):
90  data = data.data()
91  def apply_function_to_key(row):
92  row[key] = function(row[key])
93  return row
94  new_data = [apply_function_to_key(data[n]) for n in range(0, len(data))]
95  return json_data_node(new_data)
sistrip::SpyUtilities::range
const uint16_t range(const Frame &aFrame)
Definition: SiStripSpyUtilities.cc:76
data_formats.apply_function
def apply_function
Definition: data_formats.py:88
callgraph.function
string function
Definition: callgraph.py:50
def data_formats.objects_to_dicts (   script)

Definition at line 48 of file data_formats.py.

References _objects_to_dicts(), submitPVResolutionJobs.script, and str.

48 
49 def objects_to_dicts(script):
50  def new_script(self, connection):
51  try:
52  data = script(self, connection)
53  return _objects_to_dicts(data)
54  except (KeyError, TypeError) as e:
55  raise Exception("The data you gave wasn't in the correct format: %s" % str(e))
56  return new_script
57 
58 # functions used in decorators
data_formats.objects_to_dicts
def objects_to_dicts
Definition: data_formats.py:48
data_formats._objects_to_dicts
def _objects_to_dicts
Definition: data_formats.py:96
str
#define str(s)
Definition: TestProcessor.cc:53
def data_formats.query (   script)

Definition at line 39 of file data_formats.py.

References submitPVResolutionJobs.script, and str.

39 
40 def query(script):
41  def new_script(self, connection):
42  try:
43  data = script(self, connection)
44  return _to_sql_query(data)
45  except (KeyError, TypeError) as e:
46  raise Exception("The data you gave wasn't in the correct format: %s" % str(e))
47  return new_script
data_formats.query
def query
Definition: data_formats.py:39
str
#define str(s)
Definition: TestProcessor.cc:53
def data_formats.to_array_of_dicts (   script)

Definition at line 17 of file data_formats.py.

References _to_array_of_dicts(), submitPVResolutionJobs.script, and str.

17 
18 def to_array_of_dicts(script):
19  def new_script(self, connection):
20  try:
21  data = script(self, connection)
22  array_of_dicts = _to_array_of_dicts(data)
23  return json_data_node.make(array_of_dicts)
24  except (KeyError, TypeError) as e:
25  raise Exception("The data you gave wasn't in the correct format: %s" % str(e))
26  return new_script
27 
# convert {{header:value}, ..., {header:value}} to {headers:[], data:[[]]}
data_formats.to_array_of_dicts
def to_array_of_dicts
Definition: data_formats.py:17
data_formats._to_array_of_dicts
def _to_array_of_dicts
Definition: data_formats.py:59
str
#define str(s)
Definition: TestProcessor.cc:53
def data_formats.to_datatables (   script)

Definition at line 28 of file data_formats.py.

References if(), submitPVResolutionJobs.script, and str.

28 
29 def to_datatables(script):
30  def new_script(self, connection):
31  try:
32  data = script(self, connection)
33  if(type(data) == list):
34  data = _json_data_node.make(data)
35  return to_datatables(data)
36  except (KeyError, TypeError) as e:
37  raise Exception("The data you gave wasn't in the correct format: %s" % str(e))
38  return new_script
if
if(conf_.getParameter< bool >("UseStripCablingDB"))
Definition: SiStripLAProfileBooker.cc:72
str
#define str(s)
Definition: TestProcessor.cc:53
data_formats.to_datatables
def to_datatables
Definition: data_formats.py:28
def data_formats.to_sql_query (   data)

Definition at line 84 of file data_formats.py.

84 
85 def to_sql_query(data):
86  return data.to_sql()
87 
# apply function to specific column of data, assuming data
data_formats.to_sql_query
def to_sql_query
Definition: data_formats.py:84
Generated for CMSSW Reference Manual by doxygen 1.8.5