Commit cd8b61f8 authored by Avisek Naug's avatar Avisek Naug 🎨
Browse files

Initial commit

parents
Pipeline #309 canceled with stages
#ignore folders
loginfo/
ResultsPlot/
__pycache__/
.ipynb_checkpoints
.idea
ORH/
RoomTemp/
SolarData/
AlumniHallPythonVariables/
#ignore extensions
*.tfevents.*
*.zip
\ No newline at end of file
File added
%% Cell type:markdown id: tags:
# Read data on the Alumni Hall variables
* Read the data
* Remove outliers
* Remove extremely sparse data points
* NB: period denotes a 5 min interval. A period of 6 implies a timegap of 30 min. similarly a period of 12 implies a time gap of 1 hour
%% Cell type:markdown id: tags:
## 1. Preprocessing Data
%% Cell type:markdown id: tags:
### Import modules
%% Cell type:code id: tags:
``` python
from helperfunctions import *
from dataGenerator import *
from PredictionModel import *
period = 12 # ie period*5 minutes eg 12*5 60 minutes
```
%% Cell type:markdown id: tags:
### Read datafiles
%% Cell type:code id: tags:
``` python
datadirectory = 'AlumniHallPythonVariables/'
datecolumn_name = "Date / Time"
dflist1 = fileReader(datadirectory,datecolumn_name,format='%m/%d/%Y %H:%M')
datadirectory = 'ORH/'
datecolumn_name = "Date"
dflist2 = fileReader(datadirectory,datecolumn_name,format='%m/%d/%Y %H:%M')
datadirectory = 'SolarData/'
datecolumn_name ="PeriodEnd"
dflist3 = fileReader(datadirectory,datecolumn_name,format='%Y-%m-%dT%H:%M:%SZ',offset=-6)
#Subtracting offset 6 hours since looking at the data it can be inferred that it is in GMT/UTC
datadirectory = 'RoomTemp/'
datecolumn_name ="Date"
dflist4 = fileReader(datadirectory,datecolumn_name,format='%m/%d/%Y %H:%M')
```
%% Cell type:markdown id: tags:
### Merge Along rows
%% Cell type:code id: tags:
``` python
df1 = merge_df_rows(dflist1)
df2 = merge_df_rows(dflist2)
df3 = merge_df_rows(dflist3)
df4 = merge_df_rows(dflist4)
```
%% Cell type:markdown id: tags:
### Remove Outliers from desired column
%% Cell type:code id: tags:
``` python