Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Twitter API v1 stream not working with needle in Node JS (getting no data) #363

Open
frzsombor opened this issue Jul 19, 2021 · 3 comments

Comments

@frzsombor
Copy link

In my project I'm using Twitter API user timeline (both v1 and v2) and filtered stream (v2) with needle successfully, but I can't get the v1 version of filtered stream (statuses/filter) working with needle.

I feel clueless, as needle's streaming works fine with API v2 (just like the official example) and I've tested the authentication part (OAuth) of my v1 code and verified that it works well (response code: 200) - yet I'm not getting any data from the stream.

I even tried finding the problem by setting up Fiddler as a proxy of needle, but I saw nothing strange.

I've created an extracted, stand alone version of the v1 streaming module, hoping that someone can spot the issue/bug.

const uuid = require('uuid');
const oauth = require('oauth-sign');
const needle = require('needle');

const keys = {
    twitter: {
        consumerKey: '<your-consumerKey>', //API Key
        consumerSecretKey: '<your-consumerSecretKey>', //API Secret Key
        oauthToken: '<your-oauthToken>', //Access Token
        oauthTokenSecret: '<your-oauthTokenSecret>', //Access Token Secret
    }
};

const streamURL = 'https://stream.twitter.com/1.1/statuses/filter.json';

const apiParams = {
    'track': 'pizza',
};

function getAuthHeader() {
    const timestamp = Date.now() / 1000;
    const nonce = uuid.v4().replace(/-/g, '');

    let oauthParams = {
        'oauth_consumer_key': keys.twitter.consumerKey,
        'oauth_nonce': nonce,
        'oauth_signature_method': 'HMAC-SHA1',
        'oauth_timestamp': timestamp,
        'oauth_token': keys.twitter.oauthToken,
        'oauth_version': '1.0'
    };

    let mergedParams = { ...apiParams, ...oauthParams };

    oauthParams['oauth_signature'] = oauth.hmacsign(
        'POST', streamURL, mergedParams, 
        keys.twitter.consumerSecretKey, 
        keys.twitter.oauthTokenSecret
    );

    return Object.keys(oauthParams).sort().map(function (k) {
        return k + '="' + oauth.rfc3986(oauthParams[k]) + '"';
    }).join(', ');
}

function streamConnect(retryAttempt) {
    const stream = needle.post(streamURL, apiParams, {
        headers: {
            'Authorization': `OAuth ${getAuthHeader()}`,
            'Content-Type': 'application/x-www-form-urlencoded',
        },
        timeout: 20000
    });

    stream.on('header', (statusCode, headers) => {
        console.log(
            `Status: ${statusCode} ` + 
            `(${statusCode === 200 ? 'OK' : 'ERROR'})`
        );
    }).on('data', data => {
        //never receiving any data ???
        console.log('data received:');
        console.log(data);
    }).on('err', error => {
        // This reconnection logic will attempt to reconnect when a disconnection is detected.
        // To avoid rate limits, this logic implements exponential backoff, so the wait time
        // will increase if the client cannot reconnect to the stream. 
        let retryTimeout = 2 ** retryAttempt;
        console.warn(`A connection error occurred: ${error.message}. Reconnecting in ${retryTimeout/1000} seconds...`)
        setTimeout(() => {
            streamConnect(++retryAttempt);
        }, retryTimeout);
    });

    return stream;
}

streamConnect(0);
@tomas
Copy link
Owner

tomas commented Aug 27, 2021

Hi @frzsombor, what version of Node and Needle are your running? IIRC there was an issue with the nonce that was fixed a few releases ago.

@frzsombor
Copy link
Author

I'm using Node v14.15.5 and I've tried v2.9.0 and v3.0.0 too, but none of them worked with this test script.
Still don't know why, as this code works with Twitter API v2 and other libraries still work with the v1.1 API.

@mfreeman451
Copy link

Here is my code I'm using V2 of the API, hope this helps:

https://github.com/chase-app/basic-twitter-server

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants