-
Notifications
You must be signed in to change notification settings - Fork 20
Feature/convertor #14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
7cce784
0fef575
c7fa109
ff2b5a0
b267b3f
4f61268
81c7f52
d2949bc
1f12c75
59f8082
11db644
16d19cf
b107b92
64ef6df
b5210da
40fac07
0b386c4
3c9b536
c18f3ec
6a8f3d8
203ef15
2651d12
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| FROM ruby:2.7.0 | ||
|
|
||
| RUN gem install nokogiri -- --use-system-libraries | ||
|
|
||
| RUN mkdir -p /convertor | ||
| WORKDIR /convertor | ||
|
|
||
| COPY Gemfile Gemfile.lock ./ | ||
| RUN bundle install --jobs 3 | ||
|
|
||
| COPY . /convertor |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| source "https://rubygems.org" | ||
| git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } | ||
|
|
||
| ruby '2.7.0' | ||
|
|
||
| gem 'rubocop', '~> 0.80.1', require: false | ||
| gem 'nokogiri' | ||
| gem 'require_all' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| GEM | ||
| remote: https://rubygems.org/ | ||
| specs: | ||
| ast (2.4.0) | ||
| jaro_winkler (1.5.4) | ||
| mini_portile2 (2.4.0) | ||
| nokogiri (1.10.9) | ||
| mini_portile2 (~> 2.4.0) | ||
| parallel (1.19.1) | ||
| parser (2.7.0.4) | ||
| ast (~> 2.4.0) | ||
| rainbow (3.0.0) | ||
| require_all (3.0.0) | ||
| rexml (3.2.4) | ||
| rubocop (0.80.1) | ||
| jaro_winkler (~> 1.5.1) | ||
| parallel (~> 1.10) | ||
| parser (>= 2.7.0.1) | ||
| rainbow (>= 2.2.2, < 4.0) | ||
| rexml | ||
| ruby-progressbar (~> 1.7) | ||
| unicode-display_width (>= 1.4.0, < 1.7) | ||
| ruby-progressbar (1.10.1) | ||
| unicode-display_width (1.6.1) | ||
|
|
||
| PLATFORMS | ||
| ruby | ||
|
|
||
| DEPENDENCIES | ||
| nokogiri | ||
| require_all | ||
| rubocop (~> 0.80.1) | ||
|
|
||
| RUBY VERSION | ||
| ruby 2.7.0p0 | ||
|
|
||
| BUNDLED WITH | ||
| 2.1.4 |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| #!/usr/local/bin/ruby | ||
| require 'bundler/setup' | ||
| Bundler.require | ||
| require 'optparse' | ||
| require_rel '../main.rb' | ||
|
|
||
| options = {} | ||
| OptionParser.new do |opts| | ||
| opts.on("--output STRING", String) | ||
| opts.on("--input STRING", String) | ||
| opts.on("--sort STRING", String) | ||
| end.parse!(into: options) | ||
|
|
||
| main_program = Main.new(options) | ||
| main_program.run |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| version: '3.7' | ||
|
|
||
| services: | ||
| web: | ||
| build: . | ||
| volumes: | ||
| - .:/convertor:cached | ||
| - ~/.bash_history:/root/.bash_history | ||
| - bundle_cache:/bundle_cache | ||
| environment: | ||
| - BUNDLE_PATH=/bundle_cache | ||
| - GEM_HOME=/bundle_cache | ||
| - GEM_PATH=/bundle_cache | ||
| bundle_cache: | ||
| image: busybox | ||
| volumes: | ||
| - bundle_cache:/bundle_cache | ||
|
|
||
| volumes: | ||
| bundle_cache: |
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| module Converter | ||
| def self.convert(inputData, output) | ||
| nameModule = output.capitalize + 'Converter'; | ||
| Module.const_get(nameModule).convert inputData | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| module AtomConverter | ||
|
|
||
| def self.convert(inputData) | ||
| builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml| | ||
| xml.feed('xml:lang' => 'ru') { | ||
| inputData.each do |item| | ||
| xml.entry { | ||
| xml.guid item[:guid] | ||
| xml.title item[:title] | ||
| xml.description item[:description] | ||
| xml.pubDate item[:pubDate] | ||
| xml.category item[:category] | ||
| } | ||
| end | ||
| } | ||
| end | ||
| builder.to_xml | ||
| end | ||
| end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| module JsonConverter | ||
|
|
||
| def self.convert(data) | ||
| result = JSON.pretty_generate(data) | ||
| result | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| module RssConverter | ||
|
|
||
| def self.convert(inputData) | ||
| builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml| | ||
| xml.rss(version: "2.0") { | ||
| xml.channel { | ||
| inputData.each do |item| | ||
| xml.item { | ||
| xml.guid item[:guid] | ||
| xml.title item[:title] | ||
| xml.description item[:description] | ||
| xml.pubDate item[:pubDate] | ||
| xml.category item[:category] | ||
| } | ||
| end | ||
| } | ||
| } | ||
| end | ||
| builder.to_xml | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| module NameFiles | ||
| def self.find(path) | ||
| files = Dir[path] | ||
| files.map!{ |file| file.split('/').last.split('.').first } | ||
| files.map!{|name| name.split('_').map(&:capitalize).join('')} | ||
| files | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| module AtomParser | ||
|
|
||
| def self.parse(inputData) | ||
| data = [] | ||
| xml_doc = Nokogiri::XML(inputData) | ||
| xml_doc.remove_namespaces! | ||
| doc = xml_doc.xpath("//entry") | ||
| doc.each do |item| | ||
| obj = { | ||
| guid: item.at('id').text, | ||
| title: item.at('title').text, | ||
| description: item.at('summary').text, | ||
| pubDate: item.at('published').text, | ||
| category: item.at('category').text, | ||
| } | ||
| data << obj | ||
| end | ||
| data | ||
| end | ||
|
|
||
| def self.can?(data) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Не понятно, что can? Надо бы по-яснее метод описывать.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. С названиями у меня всегда проблема)) |
||
| Nokogiri::XML(data).errors.empty? && data.include?('</feed>') | ||
| end | ||
| end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| module JsonParser | ||
|
|
||
| def self.parse(inputData) | ||
| data = []; | ||
| JSON.parse(inputData).each do |item| | ||
| data << { | ||
| guid: item['guid'], | ||
| title: item['title'], | ||
| description: item['description'], | ||
| pubDate: item['pubDate'], | ||
| category: item['category'] | ||
| } | ||
| end | ||
| data | ||
| end | ||
|
|
||
| def self.can?(data) | ||
| JSON.parse(data) | ||
| return true | ||
| rescue JSON::ParserError => e | ||
| return false | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| module RssParser | ||
|
|
||
| def self.parse(data) | ||
| xml_doc = Nokogiri::XML(data) | ||
| data = []; | ||
| doc = xml_doc.xpath("//item") | ||
| doc.each do |item| | ||
| obj = { | ||
| guid: item.at('guid').text, | ||
| title: item.at('title').text, | ||
| description: item.at('description').text, | ||
| pubDate: item.at('pubDate').text, | ||
| category: item.at('category').text, | ||
| } | ||
| data << obj | ||
| end | ||
| data | ||
| end | ||
|
|
||
| def self.can?(data) | ||
| Nokogiri::XML(data).errors.empty? && !data.include?('</feed>') | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| module Parser | ||
|
|
||
| def self.parse(inputData) | ||
| filesNamed = NameFiles.find('lib/parsers/ext/*.rb') | ||
| data = [] | ||
| filesNamed.each do |nameModule| | ||
| if (Module.const_get(nameModule).can? (inputData)) | ||
| data = Module.const_get(nameModule).parse inputData | ||
| end | ||
| end | ||
| data | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| module FileReader | ||
| def self.read(input) | ||
| data = '' | ||
| open(input) { |f| | ||
| f.each_line {|line| data += line} | ||
| } | ||
| data | ||
| end | ||
|
|
||
| def self.can?(input) | ||
| File.file? input | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| module LinkReader | ||
| def self.read(input) | ||
| data = '' | ||
| OpenURI.open_uri(input) { |f| | ||
| f.each_line {|line| data += line} | ||
| } | ||
| data | ||
| end | ||
|
|
||
| def self.can?(input) | ||
| uri = URI.parse(input) | ||
| %w( http https ).include?(uri.scheme) | ||
| rescue URI::BadURIError | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. rescue URI::BadURIError, URI::InvalidURIError |
||
| false | ||
| rescue URI::InvalidURIError | ||
| false | ||
| end | ||
| end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| module Reader | ||
| def self.read(input) | ||
| files = NameFiles.find('lib/readers/ext/*.rb') | ||
|
|
||
| data = '' | ||
| files.each do |nameModule| | ||
| if (Module.const_get(nameModule).can? (input)) | ||
| data = Module.const_get(nameModule).read input | ||
| end | ||
| end | ||
| data | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| module AscSorter | ||
|
|
||
| def self.sort(data) | ||
| result = data.sort_by {|x| x[:pubDate] } | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| module WriteFile | ||
|
|
||
| def self.write(data, output) | ||
| path = 'files/file.' + output | ||
| File.open(path, 'w') do |file| | ||
| file.puts data | ||
| end | ||
| end | ||
| end |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| require 'bundler/setup' | ||
| Bundler.require | ||
| require 'nokogiri' | ||
| require 'open-uri' | ||
| require 'rubocop' | ||
| require 'json' | ||
| require 'require_all' | ||
| require 'uri' | ||
| require_all 'lib' | ||
|
|
||
| class Main | ||
| def initialize(options) | ||
| @input = options[:input] | ||
| @output = options[:output] | ||
| @sort = options[:sort] | ||
| end | ||
|
|
||
| def run | ||
| data = Reader.read(@input) | ||
| parsed_data = Parser.parse(data) | ||
| sorted_data = @sort == 'desc' ? DescSorter.sort(parsed_data) : AscSorter.sort(parsed_data) | ||
| converted_data = Converter.convert(sorted_data, @output) | ||
| WriteFile.write(converted_data, @output) | ||
| end | ||
|
|
||
| end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Видимо, код через rubocop не прогонялся..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Или он реально пустую строчку вставляет?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Не прогонялся через рубокоп, в данной реализации было важно сам функционал) учту при дальнейшей разработке.