From 5c8d0690e77e7a411bc6b797b315045068b8d3c6 Mon Sep 17 00:00:00 2001 From: "Dmitriy A. Golev" Date: Fri, 30 Oct 2015 05:10:40 +0300 Subject: [PATCH 1/3] Fix broken links for Atmosphere --- README.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/README.md b/README.md index 39a5da32..1a96ac9f 100644 --- a/README.md +++ b/README.md @@ -7,26 +7,26 @@ Demo app: ToC: ======== - - [Overview](#meteor-files) - - [Why this package?](#why-meteor-files) - - [Install](#install) - - [API](#api) - * [new Meteor.Files()](#new-meteorfilesconfig-isomorphic) [*Isomorphic*] - * [Chunk Streaming](#file-streaming) [*Example*] - * [Force Download](#file-download) [*Example*] - * [Schema](#current-schema) [*Isomorphic*] - - [Template Helper](#template-helper) - * [Force Download](#to-get-download-url-for-file-you-only-need-fileref-object-so-there-is-no-need-for-subscription) [*Example*] - * [Get file subversion](#to-get-specific-version-of-the-file-use-second-argument-version) [*Example*] - * [Thumbnail Example](#to-display-thumbnail) [*Example*] - * [Video Streaming Example](#to-stream-video) [*Example*] - - [General Methods](#methods) - * [Insert (Upload) File(s)](#insertsettings-client) [*Client*] - Upload file to server - * [Collection](#collection--isomorphic) [*Isomorphic*] - * [findOne()](#findonesearch--isomorphic) [*Isomorphic*] - * [find()](#findsearch--isomorphic) [*Isomorphic*] - * [write()](#writebuffer-options-callback--server) [*Server*] - Write binary code into FS - * [load()](#loadurl-options-callback--server) [*Server*] - Upload file from remote host + - [Overview](https://github.com/VeliovGroup/Meteor-Files#meteor-files) + - [Why this package?](https://github.com/VeliovGroup/Meteor-Files#why-meteor-files) + - [Install](https://github.com/VeliovGroup/Meteor-Files#install) + - [API](https://github.com/VeliovGroup/Meteor-Files#api) + * [new Meteor.Files()](https://github.com/VeliovGroup/Meteor-Files#new-meteorfilesconfig-isomorphic) [*Isomorphic*] + * [Chunk Streaming](https://github.com/VeliovGroup/Meteor-Files#file-streaming) [*Example*] + * [Force Download](https://github.com/VeliovGroup/Meteor-Files#file-download) [*Example*] + * [Schema](https://github.com/VeliovGroup/Meteor-Files#current-schema) [*Isomorphic*] + - [Template Helper](https://github.com/VeliovGroup/Meteor-Files#template-helper) + * [Force Download](https://github.com/VeliovGroup/Meteor-Files#to-get-download-url-for-file-you-only-need-fileref-object-so-there-is-no-need-for-subscription) [*Example*] + * [Get file subversion](https://github.com/VeliovGroup/Meteor-Files#to-get-specific-version-of-the-file-use-second-argument-version) [*Example*] + * [Thumbnail Example](https://github.com/VeliovGroup/Meteor-Files#to-display-thumbnail) [*Example*] + * [Video Streaming Example](https://github.com/VeliovGroup/Meteor-Files#to-stream-video) [*Example*] + - [General Methods](https://github.com/VeliovGroup/Meteor-Files#methods) + * [Insert (Upload) File(s)](https://github.com/VeliovGroup/Meteor-Files#insertsettings-client) [*Client*] - Upload file to server + * [Collection](https://github.com/VeliovGroup/Meteor-Files#collection--isomorphic) [*Isomorphic*] + * [findOne()](https://github.com/VeliovGroup/Meteor-Files#findonesearch--isomorphic) [*Isomorphic*] + * [find()](https://github.com/VeliovGroup/Meteor-Files#findsearch--isomorphic) [*Isomorphic*] + * [write()](https://github.com/VeliovGroup/Meteor-Files#writebuffer-options-callback--server) [*Server*] - Write binary code into FS + * [load()](https://github.com/VeliovGroup/Meteor-Files#loadurl-options-callback--server) [*Server*] - Upload file from remote host Meteor-Files ======== @@ -85,7 +85,7 @@ API - `collectionName` {*String*} - Collection name * Default value: `MeteorUploadFiles` - `cacheControl` {*String*} - Default `Cache-Control` header, by default: `public, max-age=31536000, s-maxage=31536000` - - `throttle` {*Number*} - Throttle downloads to set bps + - `throttle` {*Number* | *false*} - Throttle download speed in *bps*, by default is `false` - `downloadRoute` {*String*} - Server Route used to retrieve files * Default value: `/cdn/storage` - `schema` {*Object*} - Collection Schema (*Not editable for current release*) From 53ede40aef2e12ddbcbd06ca10911cedf316e8c3 Mon Sep 17 00:00:00 2001 From: "Dmitriy A. Golev" Date: Fri, 30 Oct 2015 06:16:06 +0300 Subject: [PATCH 2/3] v1.3.11 - Fix issue mentioned in #41 - Add throttle #40 --- .versions | 28 ++++++++++----------- files.coffee | 71 +++++++++++++++++++++++++++------------------------- package.js | 2 +- 3 files changed, 52 insertions(+), 49 deletions(-) diff --git a/.versions b/.versions index e5749ea6..bddc0578 100644 --- a/.versions +++ b/.versions @@ -10,16 +10,16 @@ boilerplate-generator@1.0.4 caching-compiler@1.0.0 caching-html-compiler@1.0.2 callback-hook@1.0.4 -check@1.0.6 -coffeescript@1.0.10 +check@1.1.0 +coffeescript@1.0.11 ddp@1.2.2 ddp-client@1.2.1 -ddp-common@1.2.1 -ddp-server@1.2.1 +ddp-common@1.2.2 +ddp-server@1.2.2 deps@1.0.9 diff-sequence@1.0.1 -ecmascript@0.1.5 -ecmascript-collections@0.1.6 +ecmascript@0.1.6 +ecmascript-runtime@0.2.6 ejson@1.0.7 geojson-utils@1.0.4 html-tools@1.0.5 @@ -36,30 +36,30 @@ iron:router@1.0.12 iron:url@1.0.11 jquery@1.11.4 logging@1.0.8 -meteor@1.1.9 +meteor@1.1.10 minifiers@1.1.7 minimongo@1.0.10 -mongo@1.1.2 +mongo@1.1.3 mongo-id@1.0.1 npm-mongo@1.4.39_1 observe-sequence@1.0.7 ordered-dict@1.0.4 ostrio:cookies@2.0.1 -ostrio:files@1.3.10 -promise@0.5.0 -random@1.0.4 -reactive-dict@1.1.2 +ostrio:files@1.3.11 +promise@0.5.1 +random@1.0.5 +reactive-dict@1.1.3 reactive-var@1.0.6 retry@1.0.4 routepolicy@1.0.6 sha@1.0.4 spacebars@1.0.7 spacebars-compiler@1.0.7 -templating@1.1.4 +templating@1.1.5 templating-tools@1.0.0 tracker@1.0.9 ui@1.0.8 underscore@1.0.4 url@1.0.5 -webapp@1.2.2 +webapp@1.2.3 webapp-hashing@1.0.5 diff --git a/files.coffee b/files.coffee index 96819afb..69b366c4 100755 --- a/files.coffee +++ b/files.coffee @@ -2,14 +2,13 @@ if Meteor.isServer ### @description Require "fs-extra" npm package ### - fs = Npm.require "fs-extra" - request = Npm.require "request" + fs = Npm.require "fs-extra" + request = Npm.require "request" Throttle = Npm.require "throttle" ### @var {object} bound - Meteor.bindEnvironment aka Fiber wrapper ### - bound = Meteor.bindEnvironment (callback) -> - return callback() + bound = Meteor.bindEnvironment (callback) -> return callback() ### @object @@ -158,9 +157,9 @@ class Meteor.Files check @schema, Object check @public, Boolean check @strict, Boolean + check @throttle, Match.OneOf false, Number check @protected, Match.OneOf Boolean, Function check @chunkSize, Number - check @throttle, Number check @permissions, Number check @storagePath, String check @downloadRoute, String @@ -253,6 +252,7 @@ class Meteor.Files throw new Meteor.Error 401, '[Meteor.Files] [remove()] Run code from client is not allowed!' _methods[self.methodNames.MeteorFileWrite] = (unitArray, fileData, meta = {}, first, chunksQty, currentChunk, totalSentChunks, randFileName, part, partsQty, fileSize) -> + @unblock() check part, Number check meta, Match.Optional Object check first, Boolean @@ -273,8 +273,6 @@ class Meteor.Files if isUploadAllowed isnt true throw new Meteor.Error(403, if _.isString(isUploadAllowed) then isUploadAllowed else "@onBeforeUpload() returned false") - @unblock() - i = 0 binary = '' while i < unitArray.byteLength @@ -301,10 +299,15 @@ class Meteor.Files result.chunk = currentChunk result.last = last - if first - fs.outputFileSync pathPart, binary, 'binary' - else - fs.appendFileSync pathPart, binary, 'binary' + try + if first + fs.outputFileSync pathPart, binary, 'binary' + else + fs.appendFileSync pathPart, binary, 'binary' + catch e + error = new Meteor.Error 500, "Unfinished upload (probably caused by server reboot)", e + console.error error + return error console.info "Meteor.Files Debugger: The part ##{part} of file #{fileName} (binary) was saved to #{pathPart}" if (chunksQty is currentChunk) and self.debug @@ -556,20 +559,20 @@ class Meteor.Files request.get(url).on('error', (error)-> throw new Meteor.Error 500, "Error on [load(#{url}, #{opts})]; Error:" + JSON.stringify error - ).on('response', (response) -> - bound -> - result = self.dataToSchema - name: fileName - path: path - meta: opts.meta - type: response.headers['content-type'] - size: response.headers['content-length'] - extension: extension - - console.info "Meteor.Files Debugger: The file #{fileName} (binary) was loaded to #{@collectionName}" if @debug + ).on('response', (response) -> bound -> - result._id = self.collection.insert _.clone result - callback and callback null, result + result = self.dataToSchema + name: fileName + path: path + meta: opts.meta + type: response.headers['content-type'] + size: response.headers['content-length'] + extension: extension + + console.info "Meteor.Files Debugger: The file #{fileName} (binary) was loaded to #{@collectionName}" if @debug + + result._id = self.collection.insert _.clone result + callback and callback null, result ).pipe fs.createOutputStream path @@ -999,10 +1002,10 @@ class Meteor.Files if partiral or (http.params.query.play and http.params.query.play == 'true') reqRange = {start, end} - if isNaN(start) and not isNaN(end) + if isNaN(start) and not isNaN end reqRange.start = end - take reqRange.end = end - if not isNaN(start) and isNaN(end) + if not isNaN(start) and isNaN end reqRange.start = start reqRange.end = start + take @@ -1050,27 +1053,27 @@ class Meteor.Files when '200' console.info "Meteor.Files Debugger: [download(#{http}, #{version})] [200]: #{fileRef.path}" if @debug stream = fs.createReadStream fileRef.path - self = @ - stream.on('open', -> + stream.on('open', => http.response.writeHead 200 - if(self.throttle) - stream.pipe(new Throttle({bps:self.throttle,chunksize:self.chunkSize})).pipe http.response + if @throttle + stream.pipe( new Throttle {bps: @throttle, chunksize: @chunkSize} + ).pipe http.response else stream.pipe http.response ).on 'error', streamErrorHandler break when '206' - self = @ console.info "Meteor.Files Debugger: [download(#{http}, #{version})] [206]: #{fileRef.path}" if @debug http.response.setHeader 'Content-Range', "bytes #{reqRange.start}-#{reqRange.end}/#{fileRef.size}" http.response.setHeader 'Content-Length', take http.response.setHeader 'Transfer-Encoding', 'chunked' - if(@throttle) + if @throttle stream = fs.createReadStream fileRef.path, {start: reqRange.start, end: reqRange.end} stream.on('open', -> http.response.writeHead 206 ).on('error', streamErrorHandler - ).on('end', -> http.response.end()) - .pipe(new Throttle({bps:self.throttle,chunksize:self.chunkSize})).pipe http.response + ).on('end', -> http.response.end() + ).pipe( new Throttle {bps: @throttle, chunksize: @chunkSize} + ).pipe http.response else stream = fs.createReadStream fileRef.path, {start: reqRange.start, end: reqRange.end} stream.on('open', -> http.response.writeHead 206 diff --git a/package.js b/package.js index 8e844a48..09e48367 100755 --- a/package.js +++ b/package.js @@ -1,6 +1,6 @@ Package.describe({ name: 'ostrio:files', - version: '1.3.10', + version: '1.3.11', summary: 'Upload, Store and Stream (Video & Audio streaming) files to/from file system (FS) via DDP and HTTP', git: 'https://github.com/VeliovGroup/Meteor-Files', documentation: 'README.md' From 416c35ba1fcc7884f764c41ec588a6937c148184 Mon Sep 17 00:00:00 2001 From: "Dmitriy A. Golev" Date: Fri, 30 Oct 2015 06:16:20 +0300 Subject: [PATCH 3/3] Demo app enhancements --- demo/client/index.coffee | 47 +++----------------------------- demo/client/index.jade | 38 ++++++-------------------- demo/client/misc/_layout.jade | 3 ++ demo/client/styles.sass | 17 +++++++++--- demo/client/upload-form.coffee | 41 ++++++++++++++++++++++++++++ demo/client/upload-form.jade | 30 ++++++++++++++++++++ demo/lib/files.collection.coffee | 7 +++-- 7 files changed, 104 insertions(+), 79 deletions(-) create mode 100644 demo/client/upload-form.coffee create mode 100644 demo/client/upload-form.jade diff --git a/demo/client/index.coffee b/demo/client/index.coffee index 60683e3c..96579550 100644 --- a/demo/client/index.coffee +++ b/demo/client/index.coffee @@ -1,9 +1,7 @@ Meteor.startup -> Template.index.onCreated -> @take = new ReactiveVar 50 - @error = new ReactiveVar false @filesLength = new ReactiveVar 0 - @uploadInstance = new ReactiveVar false @getFilesLenght = => Meteor.call 'filesLenght', (error, length) => if error @@ -15,48 +13,11 @@ Meteor.startup -> @autorun => Meteor.subscribe 'latest', @take.get() Template.index.helpers - take: -> Template.instance().take.get() - error: -> Template.instance().error.get() - latest: -> Collections.files.collection.find {}, sort: 'meta.created_at': -1 - removedIn: -> moment(@meta.expireAt).fromNow() - filesLength: -> Template.instance().filesLength.get() - uploadInstance:-> Template.instance().uploadInstance.get() - + take: -> Template.instance().take.get() + latest: -> Collections.files.collection.find {}, sort: 'meta.created_at': -1 + removedIn: -> moment(@meta.expireAt).fromNow() + filesLength: -> Template.instance().filesLength.get() Template.index.events - 'click #pause': -> @pause() - 'click #continue': -> @continue() - 'click #abort': -> @abort() 'click #loadMore': (e, template) -> template.take.set template.take.get() + 50 - 'change input[name="userfile"]': (e, template) -> template.$('form#uploadFile').submit() - 'submit form#uploadFile': (e, template) -> - e.preventDefault() - template.error.set false - files = e.currentTarget.userfile.files - - unless files.length - template.error.set "Please select a file to upload" - return false - - done = false - created_at = +new Date - template.uploadInstance.set Collections.files.insert - file: files[0] - meta: {expireAt: new Date(created_at + _app.storeTTL), created_at, downloads: 0} - onUploaded: (error, fileObj) -> - done = true - unless error - Router.go 'file', _id: fileObj._id - else - template.error.set error.reason - e.currentTarget.userfile.value = '' - template.getFilesLenght() - template.uploadInstance.set false - onAbort: -> - done = true - template.uploadInstance.set false - e.currentTarget.userfile.value = '' - onBeforeUpload: -> if @size <= 100000 * 10 * 128 then true else "Max. file size is 128MB you've tried to upload #{filesize(@size)}" - streams: 8 - false diff --git a/demo/client/index.jade b/demo/client/index.jade index 3405ce86..f05c558f 100644 --- a/demo/client/index.jade +++ b/demo/client/index.jade @@ -1,32 +1,8 @@ template(name="index") .panel.panel-default - .panel-heading: h3.panel-title Upload File - .panel-body - .center-block - if error - .alert.alert-danger {{error}} - - if uploadInstance - +with uploadInstance - .progress - .progress-bar.progress-bar-striped.active(aria-valuemin="0" aria-valuemax="100" style="width: {{progress.get}}%") - .btn-group.btn-group-justified.control-btns - if onPause.get - button#continue.btn.btn-default.btn-lg(type="button" title="Resume upload") - i.fa.fa-lg.fa-play - else - button#pause.btn.btn-default.btn-lg(type="button" title="Pause upload") - i.fa.fa-lg.fa-pause - button#abort.btn.btn-default.btn-lg(type="button" title="Abort upload") - i.fa.fa-lg.fa-stop - else - form#uploadFile - input.btn.btn-default.btn-block(title="Select File" type="file" name="userfile" required) - p.text-center: small.help Any file-type. With size less or equal to 128MB - button.btn.btn-lg.btn-primary.btn-block(type="submit" title="Upload File"): i.fa.fa-lg.fa-cloud-download - - table.table.table-bordered(style="table-layout:fixed") - tbody + .panel-heading: h3.panel-title Recently uploaded files + if latest.count + table.table.table-bordered(style="table-layout:fixed"): tbody each latest tr td: a.ellipsis(href="{{pathFor 'file' _id=_id}}") #{name} @@ -39,7 +15,9 @@ template(name="index") span.label.label-default(title="Downloads") i.fa.fa-download | #{meta.downloads} + else + .panel-body.center: .alert.alert-info Be the first to upload a file - if compare filesLength '>' latest.count - .panel-footer - button.btn.btn-default.btn-block#loadMore(type="button" title="Show older files") Load More \ No newline at end of file + if compare filesLength '>' latest.count + .panel-footer + button.btn.btn-default.btn-block#loadMore(type="button" title="Show older files") Load More \ No newline at end of file diff --git a/demo/client/misc/_layout.jade b/demo/client/misc/_layout.jade index 33d9ce65..6ab9fb47 100644 --- a/demo/client/misc/_layout.jade +++ b/demo/client/misc/_layout.jade @@ -4,5 +4,8 @@ head template(name="_layout") a(href='https://github.com/VeliovGroup/Meteor-Files') img.gh-ribbon(src='https://camo.githubusercontent.com/365986a132ccd6a44c23a9169022c0b5c890c387/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f7265645f6161303030302e706e67' alt='Fork me on GitHub' data-canonical-src='https://s3.amazonaws.com/github/ribbons/forkme_right_red_aa0000.png') + nav.navbar.navbar-default: .container-fluid + +uploadForm + .container: .row: .col-md-12 +yield \ No newline at end of file diff --git a/demo/client/styles.sass b/demo/client/styles.sass index b75a9612..1186e4d5 100644 --- a/demo/client/styles.sass +++ b/demo/client/styles.sass @@ -7,6 +7,11 @@ to opacity: 1 +.center + text-align: center + margin-right: auto + margin-left: auto + .container padding-top: 15px max-width: 640px @@ -32,10 +37,14 @@ th vertical-align: middle word-break: break-word -.control-btns > * - width: 50% !important -.panel-title - line-height: 22px +.navbar + table + width: 100% + .progress + min-width: 50vw + height: 28px + margin: 0px + .gh-ribbon position: absolute top: 0 diff --git a/demo/client/upload-form.coffee b/demo/client/upload-form.coffee new file mode 100644 index 00000000..c00bc728 --- /dev/null +++ b/demo/client/upload-form.coffee @@ -0,0 +1,41 @@ +Meteor.startup -> + Template.uploadForm.onCreated -> + @error = new ReactiveVar false + @uploadInstance = new ReactiveVar false + + Template.uploadForm.helpers + error: -> Template.instance().error.get() + uploadInstance:-> Template.instance().uploadInstance.get() + + Template.uploadForm.events + 'click #pause': -> @pause() + 'click #abort': -> @abort() + 'click #continue': -> @continue() + 'change input[name="userfile"]': (e, template) -> template.$('form#uploadFile').submit() + 'submit form#uploadFile': (e, template) -> + e.preventDefault() + template.error.set false + files = e.currentTarget.userfile.files + + unless files.length + template.error.set "Please select a file to upload" + return false + + done = false + created_at = +new Date + template.uploadInstance.set Collections.files.insert + file: files[0] + meta: {expireAt: new Date(created_at + _app.storeTTL), created_at, downloads: 0} + onUploaded: (error, fileObj) -> + done = true + unless error + Router.go 'file', _id: fileObj._id + else + template.error.set error.reason + template.uploadInstance.set false + onAbort: -> + done = true + template.uploadInstance.set false + onBeforeUpload: -> if @size <= 100000 * 10 * 128 then true else "Max. file size is 128MB you've tried to upload #{filesize(@size)}" + streams: 8 + false \ No newline at end of file diff --git a/demo/client/upload-form.jade b/demo/client/upload-form.jade new file mode 100644 index 00000000..a7cfe7fa --- /dev/null +++ b/demo/client/upload-form.jade @@ -0,0 +1,30 @@ +template(name="uploadForm") + form.navbar-form.center#uploadFile + if error + .alert.alert-danger {{error}} + + unless uploadInstance + .input-group + input.form-control.btn.btn-default(title="Select File" type="file" name="userfile" required) + span.input-group-btn + button.btn.btn-primary(type="submit" title="Upload File") + i.fa.fa-lg.fa-cloud-upload + | Upload + small.text-center.help-block Any file-type. With size less or equal to 128MB + + if uploadInstance + +with uploadInstance + table: tbody + tr + td + .btn-group + if onPause.get + button#continue.btn.btn-default.btn-sm(type="button" title="Resume upload") + i.fa.fa-fw.fa-play + else + button#pause.btn.btn-default.btn-sm(type="button" title="Pause upload") + i.fa.fa-fw.fa-pause + button#abort.btn.btn-default.btn-sm(type="button" title="Abort upload") + i.fa.fa-fw.fa-stop + td: .progress.center: .progress-bar.progress-bar-striped.active(aria-valuemin="0" aria-valuemax="100" style="width: {{progress.get}}%") + tr: td.center(colspan="2"): small.text-center.help-block(style="margin-bottom:0px") You are free to browse the site while upload in progress \ No newline at end of file diff --git a/demo/lib/files.collection.coffee b/demo/lib/files.collection.coffee index 2d741f5f..a35891cb 100644 --- a/demo/lib/files.collection.coffee +++ b/demo/lib/files.collection.coffee @@ -1,10 +1,11 @@ Collections.files = new Meteor.Files debug: false + throttle: 256*256*64 + chunkSize: 256*256*4 storagePath: 'assets/app/uploads/uploadedFiles' collectionName: 'uploadedFiles' - chunkSize: 256*256*8 - onBeforeUpload: -> if @size <= 100000 * 10 * 128 then true else "Max. file size is 128MB you've tried to upload #{filesize(@size)}" allowClientCode: false + onBeforeUpload: -> if @size <= 100000 * 10 * 128 then true else "Max. file size is 128MB you've tried to upload #{filesize(@size)}" downloadCallback: (fileObj) -> if @params?.query.download is 'true' Collections.files.collection.update fileObj._id, $inc: 'meta.downloads': 1 @@ -17,6 +18,8 @@ if Meteor.isServer Collections.files.collection._ensureIndex {'meta.expireAt': 1}, {expireAfterSeconds: 0, background: true} + Meteor.startup -> Collections.files.remove {} + Meteor.publish 'latest', (take = 50)-> check take, Number Collections.files.collection.find {}