Skip to content

Commit 156de0d

Browse files
author
Brigid Goulem
committed
Step 4: Process logs and generate inventory.txt
1 parent 4446d15 commit 156de0d

538 files changed

Lines changed: 735 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎newproject/README.md‎

‎newproject/analysis/main.py‎

Whitespace-only changes.

‎newproject/assignment.sh‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
#!/bin/bash
2+
set -x
3+
4+
############################################
5+
# DSI CONSULTING INC. Project setup script #
6+
############################################
7+
# This script creates standard analysis and output directories
8+
# for a new project. It also creates a README file with the
9+
# project name and a brief description of the project.
10+
# Then it unzips the raw data provided by the client.
11+
12+
if [ -d newproject ]; then
13+
echo "Recreating the newproject directory"
14+
rm -rf newproject
15+
fi
16+
mkdir newproject
17+
cd newproject
18+
19+
mkdir analysis output
20+
touch README.md
21+
touch analysis/main.py
22+
23+
# download client data
24+
curl -Lo rawdata.zip https://github.com/UofT-DSI/shell/raw/refs/heads/main/02_activities/assignments/rawdata.zip
25+
unzip -q rawdata.zip
26+
27+
###########################################
28+
# Complete assignment here
29+
30+
# 1. Create a directory named data
31+
32+
# 2. Move the ./rawdata directory to ./data/raw (eg. move it into ./data and rename it to raw)
33+
34+
# 3. List the contents of the ./data/raw directory
35+
36+
# 4. Create the directory ./data/processed,
37+
# then create the following sub-directories within it: server_logs, user_logs, and event_logs
38+
39+
# 5. Copy all server log files (files with "server" in the name AND a .log extension) from ./data/raw to ./data/processed/server_logs
40+
41+
# 6. Repeat the above step for user logs and event logs
42+
43+
# 7. For user privacy, remove all files containing IP addresses (files with "ipaddr" in the filename) from ./data/raw and ./data/processed/user_logs
44+
45+
# 8. Create a file named ./data/inventory.txt that lists all the files in the subfolders of ./data/processed
46+
47+
48+
###########################################
49+
50+
echo "Project setup is complete!"

‎newproject/data/inventory.txt‎

Lines changed: 155 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
Server Logs:
2+
data/processed/server_logs/server_log_11.log
3+
data/processed/server_logs/server_log_39.log
4+
data/processed/server_logs/server_log_38.log
5+
data/processed/server_logs/server_log_10.log
6+
data/processed/server_logs/server_log_12.log
7+
data/processed/server_logs/server_log_13.log
8+
data/processed/server_logs/server_log_17.log
9+
data/processed/server_logs/server_log_16.log
10+
data/processed/server_logs/server_log_28.log
11+
data/processed/server_logs/server_log_14.log
12+
data/processed/server_logs/server_log_15.log
13+
data/processed/server_logs/server_log_29.log
14+
data/processed/server_logs/server_log_3.log
15+
data/processed/server_logs/server_log_2.log
16+
data/processed/server_logs/server_log_1.log
17+
data/processed/server_logs/server_log_48.log
18+
data/processed/server_logs/server_log_5.log
19+
data/processed/server_logs/server_log_4.log
20+
data/processed/server_logs/server_log_49.log
21+
data/processed/server_logs/server_log_6.log
22+
data/processed/server_logs/server_log_7.log
23+
data/processed/server_logs/server_log_47.log
24+
data/processed/server_logs/server_log_46.log
25+
data/processed/server_logs/server_log_44.log
26+
data/processed/server_logs/server_log_50.log
27+
data/processed/server_logs/server_log_9.log
28+
data/processed/server_logs/server_log_8.log
29+
data/processed/server_logs/server_log_45.log
30+
data/processed/server_logs/server_log_41.log
31+
data/processed/server_logs/server_log_40.log
32+
data/processed/server_logs/server_log_42.log
33+
data/processed/server_logs/server_log_43.log
34+
data/processed/server_logs/server_log_30.log
35+
data/processed/server_logs/server_log_24.log
36+
data/processed/server_logs/server_log_18.log
37+
data/processed/server_logs/server_log_19.log
38+
data/processed/server_logs/server_log_25.log
39+
data/processed/server_logs/server_log_31.log
40+
data/processed/server_logs/server_log_27.log
41+
data/processed/server_logs/server_log_33.log
42+
data/processed/server_logs/server_log_32.log
43+
data/processed/server_logs/server_log_26.log
44+
data/processed/server_logs/server_log_22.log
45+
data/processed/server_logs/server_log_36.log
46+
data/processed/server_logs/server_log_37.log
47+
data/processed/server_logs/server_log_23.log
48+
data/processed/server_logs/server_log_35.log
49+
data/processed/server_logs/server_log_21.log
50+
data/processed/server_logs/server_log_20.log
51+
data/processed/server_logs/server_log_34.log
52+
53+
User Logs:
54+
data/processed/user_logs/user_log_33.log
55+
data/processed/user_logs/user_log_27.log
56+
data/processed/user_logs/user_log_26.log
57+
data/processed/user_logs/user_log_32.log
58+
data/processed/user_logs/user_log_18.log
59+
data/processed/user_logs/user_log_24.log
60+
data/processed/user_logs/user_log_30.log
61+
data/processed/user_logs/user_log_31.log
62+
data/processed/user_logs/user_log_25.log
63+
data/processed/user_logs/user_log_19.log
64+
data/processed/user_logs/user_log_21.log
65+
data/processed/user_logs/user_log_35.log
66+
data/processed/user_logs/user_log_34.log
67+
data/processed/user_logs/user_log_20.log
68+
data/processed/user_logs/user_log_36.log
69+
data/processed/user_logs/user_log_22.log
70+
data/processed/user_logs/user_log_23.log
71+
data/processed/user_logs/user_log_37.log
72+
data/processed/user_logs/user_log_50.log
73+
data/processed/user_logs/user_log_44.log
74+
data/processed/user_logs/user_log_45.log
75+
data/processed/user_logs/user_log_47.log
76+
data/processed/user_logs/user_log_46.log
77+
data/processed/user_logs/user_log_42.log
78+
data/processed/user_logs/user_log_43.log
79+
data/processed/user_logs/user_log_41.log
80+
data/processed/user_logs/user_log_9.log
81+
data/processed/user_logs/user_log_8.log
82+
data/processed/user_logs/user_log_40.log
83+
data/processed/user_logs/user_log_5.log
84+
data/processed/user_logs/user_log_4.log
85+
data/processed/user_logs/user_log_6.log
86+
data/processed/user_logs/user_log_7.log
87+
data/processed/user_logs/user_log_3.log
88+
data/processed/user_logs/user_log_2.log
89+
data/processed/user_logs/user_log_48.log
90+
data/processed/user_logs/user_log_1.log
91+
data/processed/user_logs/user_log_49.log
92+
data/processed/user_logs/user_log_12.log
93+
data/processed/user_logs/user_log_13.log
94+
data/processed/user_logs/user_log_39.log
95+
data/processed/user_logs/user_log_11.log
96+
data/processed/user_logs/user_log_10.log
97+
data/processed/user_logs/user_log_38.log
98+
data/processed/user_logs/user_log_14.log
99+
data/processed/user_logs/user_log_28.log
100+
data/processed/user_logs/user_log_29.log
101+
data/processed/user_logs/user_log_15.log
102+
data/processed/user_logs/user_log_17.log
103+
data/processed/user_logs/user_log_16.log
104+
105+
Event Logs:
106+
data/processed/event_logs/event_log_48.log
107+
data/processed/event_logs/event_log_49.log
108+
data/processed/event_logs/event_log_8.log
109+
data/processed/event_logs/event_log_9.log
110+
data/processed/event_logs/event_log_14.log
111+
data/processed/event_logs/event_log_28.log
112+
data/processed/event_logs/event_log_29.log
113+
data/processed/event_logs/event_log_15.log
114+
data/processed/event_logs/event_log_17.log
115+
data/processed/event_logs/event_log_16.log
116+
data/processed/event_logs/event_log_12.log
117+
data/processed/event_logs/event_log_13.log
118+
data/processed/event_logs/event_log_39.log
119+
data/processed/event_logs/event_log_11.log
120+
data/processed/event_logs/event_log_10.log
121+
data/processed/event_logs/event_log_38.log
122+
data/processed/event_logs/event_log_35.log
123+
data/processed/event_logs/event_log_21.log
124+
data/processed/event_logs/event_log_20.log
125+
data/processed/event_logs/event_log_34.log
126+
data/processed/event_logs/event_log_22.log
127+
data/processed/event_logs/event_log_36.log
128+
data/processed/event_logs/event_log_37.log
129+
data/processed/event_logs/event_log_23.log
130+
data/processed/event_logs/event_log_27.log
131+
data/processed/event_logs/event_log_33.log
132+
data/processed/event_logs/event_log_32.log
133+
data/processed/event_logs/event_log_26.log
134+
data/processed/event_logs/event_log_18.log
135+
data/processed/event_logs/event_log_30.log
136+
data/processed/event_logs/event_log_24.log
137+
data/processed/event_logs/event_log_25.log
138+
data/processed/event_logs/event_log_31.log
139+
data/processed/event_logs/event_log_19.log
140+
data/processed/event_logs/event_log_42.log
141+
data/processed/event_logs/event_log_7.log
142+
data/processed/event_logs/event_log_6.log
143+
data/processed/event_logs/event_log_43.log
144+
data/processed/event_logs/event_log_41.log
145+
data/processed/event_logs/event_log_4.log
146+
data/processed/event_logs/event_log_5.log
147+
data/processed/event_logs/event_log_40.log
148+
data/processed/event_logs/event_log_44.log
149+
data/processed/event_logs/event_log_50.log
150+
data/processed/event_logs/event_log_1.log
151+
data/processed/event_logs/event_log_45.log
152+
data/processed/event_logs/event_log_47.log
153+
data/processed/event_logs/event_log_2.log
154+
data/processed/event_logs/event_log_3.log
155+
data/processed/event_logs/event_log_46.log

‎newproject/data/processed/event_logs/event_log_1.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 1

‎newproject/data/processed/event_logs/event_log_10.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 10

‎newproject/data/processed/event_logs/event_log_11.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 11

‎newproject/data/processed/event_logs/event_log_12.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 12

‎newproject/data/processed/event_logs/event_log_13.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 13

‎newproject/data/processed/event_logs/event_log_14.log‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Event log data 14

0 commit comments

Comments
 (0)