My initial requirement is to track/find changes in directories over time, but without duplicating it or without having direct access to it, and for the background, I need this to help me compute deltas for an incremental backup module (incremental-backups-tools).

(If you want to compare local directories, you should take a look at the filecmp module.)

My approach is to take a "snapshot" / compute an "index" of a directory which will contains:

list of files
list of subdirectories
a mapping filename => last modified time or the file hash

You can easily dump it as JSON.

Later, when you want to compute changes, you just need to compare the previous index, with the current one, and track:

deleted files
new files
updated files
deleted subdirectories

I create a directory for testing:

!rm -rf /tmp/test_tracking
!mkdir /tmp/test_tracking
!mkdir /tmp/test_tracking/subdir1
!mkdir /tmp/test_tracking/subdir2
!echo content1 > /tmp/test_tracking/file1
!echo content2 > /tmp/test_tracking/file2
!echo content3 > /tmp/test_tracking/subdir1/file3
!echo content4 > /tmp/test_tracking/subdir2/file4
!ls -lR /tmp/test_tracking/

/tmp/test_tracking/:
total 16
-rw-rw-r-- 1 thomas thomas    9 déc.  12 21:45 file1
-rw-rw-r-- 1 thomas thomas    9 déc.  12 21:45 file2
drwxrwxr-x 2 thomas thomas 4096 déc.  12 21:45 subdir1
drwxrwxr-x 2 thomas thomas 4096 déc.  12 21:45 subdir2

/tmp/test_tracking/subdir1:
total 4
-rw-rw-r-- 1 thomas thomas 9 déc.  12 21:45 file3

/tmp/test_tracking/subdir2:
total 4
-rw-rw-r-- 1 thomas thomas 9 déc.  12 21:45 file4

Files and subdir

First, we need to get a list of files and subdirectories relative to the root, using os.walk is the easiest way.

import os

path = '/tmp/test_tracking/'
files = []
subdirs = []

for root, dirs, filenames in os.walk(path):
    for subdir in dirs:
        subdirs.append(os.path.relpath(os.path.join(root, subdir), path))

    for f in filenames:
        files.append(os.path.relpath(os.path.join(root, f), path))

We check the result:

files

['file2', 'file1', 'subdir2/file4', 'subdir1/file3']

subdirs

['subdir2', 'subdir1']

Create an index

Once we have the list of files and subdirectories, we need to create the actual index, which will help us detect updated files.

There is two ways to check if a file has been modified that meet the requirement, the slow way: compute a hash (md5, sha256...) and the fast way: rely on the last modified time (even rsync relies on last modified time by default).

Last modified time

Using os.path:

file_mtime = os.path.getmtime(os.path.join(path, files[0]))

We can convert it easily to a datetime object:

from datetime import datetime
file_mtime, datetime.fromtimestamp(file_mtime)

(1386881105.388609, datetime.datetime(2013, 12, 12, 21, 45, 5, 388609))

Sha256 hash

Using hashlib:

import hashlib

def filehash(filepath, blocksize=4096):
    """ Return the hash hexdigest for the file `filepath', processing the file
    by chunk of `blocksize'.

    :type filepath: str
    :param filepath: Path to file

    :type blocksize: int
    :param blocksize: Size of the chunk when processing the file

    """
    sha = hashlib.sha256()
    with open(filepath, 'rb') as fp:
        while 1:
            data = fp.read(blocksize)
            if data:
                sha.update(data)
            else:
                break
    return sha.hexdigest()

file_hash = filehash(os.path.join(path, files[0]))

file_hash

'e0763097d2327a89fb7fc6a1fad40f87d2261dcdd6c09e65ee00b200a0128e1c'

Since I don't want to spend hours computing changes, I often choose the mtime. We can build an index, basically a dictionnary, that maps filepath (relative to its own root) to its last modified time.

index = {}
for f in files:
    index[f] = os.path.getmtime(os.path.join(path, files[0]))

index[files[0]]

1386881105.388609

Final index

So to be able to find differences between directory we need:

The list of files
A list of every subdirectory
And a index (last mtime or hash)

import os

def compute_dir_index(path):
    """ Return a tuple containing:
    - list of files (relative to path)
    - lisf of subdirs (relative to path)
    - a dict: filepath => last 
    """
    files = []
    subdirs = []

    for root, dirs, filenames in os.walk(path):
        for subdir in dirs:
            subdirs.append(os.path.relpath(os.path.join(root, subdir), path))

        for f in filenames:
            files.append(os.path.relpath(os.path.join(root, f), path))
        
    index = {}
    for f in files:
        index[f] = os.path.getmtime(os.path.join(path, files[0]))

    return dict(files=files, subdirs=subdirs, index=index)

diff = compute_dir_index(path)

Computing changes

I make some changes and recompute a new diff:

!rm -rf /tmp/test_tracking/subdir2
!echo newcontent1 > /tmp/test_tracking/file1
!echo content5 > /tmp/test_tracking/file5
!ls -lR /tmp/test_tracking/

/tmp/test_tracking/:
total 16
-rw-rw-r-- 1 thomas thomas   12 déc.  12 21:45 file1
-rw-rw-r-- 1 thomas thomas    9 déc.  12 21:45 file2
-rw-rw-r-- 1 thomas thomas    9 déc.  12 21:45 file5
drwxrwxr-x 2 thomas thomas 4096 déc.  12 21:45 subdir1

/tmp/test_tracking/subdir1:
total 4
-rw-rw-r-- 1 thomas thomas 9 déc.  12 21:45 file3

diff2 = compute_dir_index(path)

Computing the difference is easy, we need few things actually:

Deleted files (just a set substraction)
New files (just a set substraction)
Updated files (thanks to the index)
Deleted subdirectories

Here is the compute_diff:

def compute_diff(dir_base, dir_cmp):
    data = {}
    data['deleted'] = list(set(dir_cmp['files']) - set(dir_base['files']))
    data['created'] = list(set(dir_base['files']) - set(dir_cmp['files']))
    data['updated'] = []
    data['deleted_dirs'] = list(set(dir_cmp['subdirs']) - set(dir_base['subdirs']))

    for f in set(dir_cmp['files']).intersection(set(dir_base['files'])):
        if dir_base['index'][f] != dir_cmp['index'][f]:
            data['updated'].append(f)

    return data

We can check with our previously created diffs:

compute_diff(diff2, diff)

{'created': ['file5'],
 'deleted': ['subdir2/file4'],
 'deleted_dirs': ['subdir2'],
 'updated': ['file2', 'file1', 'subdir1/file3']}

Dirtools

I recently added this feature in Dirtools, pull requests and feedback are welcome!

from dirtools import Dir, DirState

d = Dir(path)
dir_state = DirState(d)

state_file = dir_state.to_json()

# Later... after some changes

dir_state = DirState.from_json(state_file)
dir_state2 = DirState(d)

changes = dir_state2 - dir_state

Your feedback

Please don't hesitate if you have any questions, ideas, or improvements!

I recently received an ASIC miner, and I use the popular cgminer, an ASIC miner (written in C) for Bitcoin (and Litecoin).

Cgminer provides an RPC API that let you retrieve stats about your device(s), and I'm buidling an open source monitoring dashboard, RigsMonitoring writen in Python, accessing the API with Python is easy using sockets.

Enable cgminer API

To enable the API, you need to add --api-listen and --api-allow W:0/0.

sudo nohup ./cgminer -o stratum.bitcoin.cz:3333 -u myworker -p mypass --api-allow W:0/0 --api-listen&

Accessing the API with Python

A detailled documentation is available in the API-README file.

The API talks JSON, listens on port 4048 by default, and a basic request looks like {"command":"CMD","parameter":"PARAM"}.

import socket
import json

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('localhost', 4028))

Make a request with the summary command, JSON encoded.

sock.send(json.dumps({'command': 'summary'}))

22

Next, receive the response (alway JSON encoded):

resp = ''
while 1:
    buf = sock.recv(4096)
    if buf:
        resp += buf
    else:
        break

resp

'{"STATUS":[{"STATUS":"S","When":1378733100,"Code":11,"Msg":"Summary","Description":"cgminer 3.3.4"}],"SUMMARY":[{"Elapsed":8061,"MHS av":5363.24,"Found Blocks":0,"Getworks":303,"Accepted":3379,"Rejected":16,"Hardware Errors":1,"Utility":25.15,"Discarded":604,"Stale":0,"Get Failures":0,"Local Work":12271,"Remote Failures":0,"Network Blocks":23,"Total MH":43233140.8015,"Work Utility":75.47,"Difficulty Accepted":10137.00000000,"Difficulty Rejected":48.00000000,"Difficulty Stale":0.00000000,"Best Share":8944,"Device Hardware%":0.0099,"Device Rejected%":0.4734,"Pool Rejected%":0.4713,"Pool Stale%":0.0000}],"id":1}\x00'

You can notice the null byte \x00 at the end of the response.

Finally, we shutdown the socket.

sock.shutdown(socket.SHUT_RDWR)
sock.close()

And decode the response (without the null byte):

json.loads(resp[:-1])

{u'STATUS': [{u'Code': 11,
   u'Description': u'cgminer 3.3.4',
   u'Msg': u'Summary',
   u'STATUS': u'S',
   u'When': 1378733100}],
 u'SUMMARY': [{u'Accepted': 3379,
   u'Best Share': 8944,
   u'Device Hardware%': 0.0099,
   u'Device Rejected%': 0.4734,
   u'Difficulty Accepted': 10137.0,
   u'Difficulty Rejected': 48.0,
   u'Difficulty Stale': 0.0,
   u'Discarded': 604,
   u'Elapsed': 8061,
   u'Found Blocks': 0,
   u'Get Failures': 0,
   u'Getworks': 303,
   u'Hardware Errors': 1,
   u'Local Work': 12271,
   u'MHS av': 5363.24,
   u'Network Blocks': 23,
   u'Pool Rejected%': 0.4713,
   u'Pool Stale%': 0.0,
   u'Rejected': 16,
   u'Remote Failures': 0,
   u'Stale': 0,
   u'Total MH': 43233140.8015,
   u'Utility': 25.15,
   u'Work Utility': 75.47}],
 u'id': 1}

Cgminer API wrapper

Now that we successfully called the API, here is a basic wrapper, with argument support:

import socket
import json


class CgminerAPI(object):
    """ Cgminer RPC API wrapper. """
    def __init__(self, host='localhost', port=4028):
        self.data = {}
        self.host = host
        self.port = port

    def command(self, command, arg=None):
        """ Initialize a socket connection,
        send a command (a json encoded dict) and
        receive the response (and decode it).
        """
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

        try:
            sock.connect((self.host, self.port))
            payload = {"command": command}
            if arg is not None:
                # Parameter must be converted to basestring (no int)
                payload.update({'parameter': unicode(arg)})

            sock.send(json.dumps(payload))
            received = self._receive(sock)
        finally:
            sock.shutdown(socket.SHUT_RDWR)
            sock.close()
        
        return json.loads(received[:-1])

    def _receive(self, sock, size=4096):
        msg = ''
        while 1:
            chunk = sock.recv(size)
            if chunk:
                msg += chunk
            else:
                break
        return msg

    def __getattr__(self, attr):
        def out(arg=None):
            return self.command(attr, arg)
        return out

Check that the wrapper is actually working, by calling the command method:

cgminer = CgminerAPI()

cgminer.command('summary')

{u'STATUS': [{u'Code': 11,
   u'Description': u'cgminer 3.3.4',
   u'Msg': u'Summary',
   u'STATUS': u'S',
   u'When': 1378918507}],
 u'SUMMARY': [{u'Accepted': 69859,
   u'Best Share': 1498416,
   u'Device Hardware%': 0.0005,
   u'Device Rejected%': 0.3646,
   u'Difficulty Accepted': 209577.0,
   u'Difficulty Rejected': 768.0,
   u'Difficulty Stale': 0.0,
   u'Discarded': 12286,
   u'Elapsed': 193470,
   u'Found Blocks': 0,
   u'Get Failures': 2,
   u'Getworks': 6147,
   u'Hardware Errors': 1,
   u'Local Work': 259407,
   u'MHS av': 4676.09,
   u'Network Blocks': 371,
   u'Pool Rejected%': 0.3651,
   u'Pool Stale%': 0.0,
   u'Rejected': 256,
   u'Remote Failures': 1,
   u'Stale': 3,
   u'Total MH': 904683321.2956,
   u'Utility': 21.67,
   u'Work Utility': 65.32}],
 u'id': 1}

Have you noticed the __getattr__ magic method, it return the command so instead of calling the command method, we can call the method that correspond to the command we want, it allows us to:

cgminer.summary()

{u'STATUS': [{u'Code': 11,
   u'Description': u'cgminer 3.3.4',
   u'Msg': u'Summary',
   u'STATUS': u'S',
   u'When': 1378918691}],
 u'SUMMARY': [{u'Accepted': 69925,
   u'Best Share': 1498416,
   u'Device Hardware%': 0.0005,
   u'Device Rejected%': 0.3671,
   u'Difficulty Accepted': 209775.0,
   u'Difficulty Rejected': 774.0,
   u'Difficulty Stale': 0.0,
   u'Discarded': 12302,
   u'Elapsed': 193655,
   u'Found Blocks': 0,
   u'Get Failures': 2,
   u'Getworks': 6155,
   u'Hardware Errors': 1,
   u'Local Work': 259678,
   u'MHS av': 4676.69,
   u'Network Blocks': 372,
   u'Pool Rejected%': 0.3676,
   u'Pool Stale%': 0.0,
   u'Rejected': 258,
   u'Remote Failures': 1,
   u'Stale': 3,
   u'Total MH': 905662573.8391,
   u'Utility': 21.66,
   u'Work Utility': 65.33}],
 u'id': 1}

And it even handles argument:

cgminer.asc(0)

{u'ASC': [{u'ASC': 0,
   u'Accepted': 69928,
   u'Device Hardware%': 0.0005,
   u'Device Rejected%': 0.367,
   u'Diff1 Work': 210881,
   u'Difficulty Accepted': 209784.0,
   u'Difficulty Rejected': 774.0,
   u'Enabled': u'Y',
   u'Hardware Errors': 1,
   u'ID': 0,
   u'Last Share Difficulty': 3.0,
   u'Last Share Pool': 0,
   u'Last Share Time': 1378918702,
   u'Last Valid Work': 1378918705,
   u'MHS 5s': 5412.69,
   u'MHS av': 4676.78,
   u'Name': u'BAJ',
   u'No Device': False,
   u'Rejected': 258,
   u'Status': u'Alive',
   u'Temperature': 35.0,
   u'Total MH': 905752768.1523,
   u'Utility': 21.66}],
 u'STATUS': [{u'Code': 106,
   u'Description': u'cgminer 3.3.4',
   u'Msg': u'ASC0',
   u'STATUS': u'S',
   u'When': 1378918707}],
 u'id': 1}

Pycgminer

I released the code on pypi and you can check out the project on github: pycgminer.

$ pip install pycgminer

from pycgminer import CgminerAPI
    
cgminer = CgminerAPI()
print cgminer.devs()

{u'STATUS': [{u'STATUS': u'S', u'Msg': u'1 ASC(s) - ', u'Code': 9, u'When': 1379356749, u'Description': u'cgminer 3.3.4'}], u'DEVS': [{u'Difficulty Accepted': 755613.0, u'Temperature': 36.0, u'Difficulty Rejected': 3070.0, u'Status': u'Alive', u'Device Rejected%': 0.4048, u'Rejected': 942, u'ID': 0, u'ASC': 0, u'Hardware Errors': 7, u'Accepted': 231709, u'No Device': False, u'Last Share Pool': 0, u'Diff1 Work': 758437, u'Name': u'BAJ', u'Total MH': 3255572325.457, u'Enabled': u'Y', u'Device Hardware%': 0.0009, u'Last Valid Work': 1379356749, u'Last Share Time': 1379356749, u'MHS av': 5153.57, u'MHS 5s': 5380.7, u'Last Share Difficulty': 4.0, u'Utility': 22.01}], u'id': 1}

Don't hesitate to submit a pull request!

Feedback

Don't hesitate to let me know if you have any questions/suggestions !

If like me, you want to backup everything you ever published, it makes sense to backup your tweets. Camlistore, also known as personal storage system for life seems a perfect fit for the job.

I chose to use Python since I recently created a Python client for Camlistore, Camlipy and I already played with Twitter API using requests (see using Twitter API v1.1 with Python).

Requirements

$ sudo pip install request request_oauthlib camlipy

Twitter API

Here is the Twitter API OAuth part, from my previous article, you must set your API keys and run the script to generate your OAuth tokens.

import json
from requests_oauthlib import OAuth1
from urlparse import parse_qs
import sys
import datetime
import locale

import requests

REQUEST_TOKEN_URL = "https://api.twitter.com/oauth/request_token"
AUTHORIZE_URL = "https://api.twitter.com/oauth/authorize?oauth_token="
ACCESS_TOKEN_URL = "https://api.twitter.com/oauth/access_token"

CONSUMER_KEY = "XXXXXXXXX"
CONSUMER_SECRET = "XXXXXXXXX"

OAUTH_TOKEN = ""
OAUTH_TOKEN_SECRET = ""

TWITTER_API_TIMELINE = 'https://api.twitter.com/1.1/statuses/user_timeline.json'


def setup_oauth():
    """Authorize your app via identifier."""
    # Request token
    oauth = OAuth1(CONSUMER_KEY, client_secret=CONSUMER_SECRET)
    r = requests.post(url=REQUEST_TOKEN_URL, auth=oauth)
    credentials = parse_qs(r.content)

    resource_owner_key = credentials.get('oauth_token')[0]
    resource_owner_secret = credentials.get('oauth_token_secret')[0]

    # Authorize
    authorize_url = AUTHORIZE_URL + resource_owner_key
    print 'Please go here and authorize: ' + authorize_url

    verifier = raw_input('Please input the verifier: ')
    oauth = OAuth1(CONSUMER_KEY,
                   client_secret=CONSUMER_SECRET,
                   resource_owner_key=resource_owner_key,
                   resource_owner_secret=resource_owner_secret,
                   verifier=verifier)

    # Finally, Obtain the Access Token
    r = requests.post(url=ACCESS_TOKEN_URL, auth=oauth)
    credentials = parse_qs(r.content)
    token = credentials.get('oauth_token')[0]
    secret = credentials.get('oauth_token_secret')[0]

    return token, secret


def get_oauth():
    oauth = OAuth1(CONSUMER_KEY,
                   client_secret=CONSUMER_SECRET,
                   resource_owner_key=OAUTH_TOKEN,
                   resource_owner_secret=OAUTH_TOKEN_SECRET)
    return oauth

if not OAUTH_TOKEN:
    token, secret = setup_oauth()
    print "OAUTH_TOKEN: " + token
    print "OAUTH_TOKEN_SECRET: " + secret
    print
    sys.exit()

Now we can call the Twitter API, we'll hit the statuses/user_timeline endpoint (check out working with timelines if needed).

def fetch_tweets(since_id=None, max_id=None, count=200):
    """ Fetch tweets. """
    params = {'count': count}
    if since_id:
        params['since_id'] = since_id
    if max_id:
        params['max_id'] = max_id

    oauth = get_oauth()
    r = requests.get(url=TWITTER_API_TIMELINE,
                     params=params, auth=oauth)
    return r.json()

Here is the backup_timeline function that automatically handle the max_id paramaters since we can only retrieve 200 tweets by API call.

def backup_timeline(since_id=None, max_id=None, count=200):
    tweets = []
    while 1:
        batch = fetch_tweets(since_id=since_id, max_id=max_id, count=count)
        tweets.extend(batch)
        if tweets:
            max_id = tweets[-1]['id']
        if len(batch) < count:
            break
    return tweets

Now we can fetch our timeline (it will return only the first 3200 tweets, this is the API limit).

tweets = backup_timeline()

And we need to retrieve the most recent id for the next time we will backup.

since_id = tweets[0]['id']

Check that backup_timeline result is empty with since_id.

backup_timeline(since_id=since_id)

[]

Getting started with Camlistore and Camlipy

If you have never heard about Camlistore before, you should read the website, and the project overview. You should also check out Camlipy documentation.

I am assuming you have at least a little knowledge about how Camlistore works.

Each blob is identified by its unique blobref. A blobref looks like sha1-25f2b42fae7398bc8857ed17d56d7d1e072c9832.

from camlipy import Camlistore
c = Camlistore('http://localhost:3179')

First, we need to create a permanode, it will hold the static set blobref reference, since we will create a new static set each time we perform a backup.

p = c.permanode_by_title('twitter_backups', create=True)

p

<Schema Permanode:sha1-25f2b42fae7398bc8857ed17d56d7d1e072c9832>

Next, we create a new static set, which will contain each tweets blobrefs (most recent first), with the add_to_static_set helper, and we add this set to a set.

s = c.add_to_static_set([c.put_blob(json.dumps(tweet)) for tweet in tweets])
p.set_camli_content(c.add_to_static_set([s]))

Finally we can store the newest id, since_id for the next search, in a permanode attribute (permanode attribute must be str), so we create a new claim.

p.set_attr('since_id', str(since_id))

p.get_attr('since_id')

u'369579788119199746'

The final script

Without Twitter API code.

# Fetch the permanode
p = c.permanode_by_title('twitter_backups')

# Retrieve since_id if existing
since_id = int(p.get_attr('since_id'))

# Try to retrieve the existing static set
static_set_blobref = p.get_camli_content()

# A static set already exists, so we load it
if static_set_blobref:
    static_set = c.static_set(static_set_blobref)
else:
    # We create a new static set
    static_set = c.static_set()

# Update/set the new tweets blobrefs
tweets_blobrefs = [c.put_blob(json.dumps(tweet)) for tweet in backup_timeline(since_id=since_id)]
if tweets_blobrefs:
    new_static_set_blobref = static_set.update([c.add_to_static_set(tweets_blobrefs)])
    p.set_camli_content(new_static_set_blobref)

Accessing tweets

p2 = c.permanode_by_title('twitter_backups', create=True)

s2 = c.static_set(p2.get_camli_content())

To retrieve the tweets starting from the most recent, we must iterate the list upside down.

tweets = []
for static_set_blobref in s2.members[::-1]:
    static_set = c.static_set(static_set_blobref)
    for tweet_data in static_set.members:
        tweet = json.loads(c.get_blob(tweet_data).read())
        tweets.append(tweet)

tweets[0]

{u'contributors': None,
 u'coordinates': None,
 u'created_at': u'Mon Aug 19 22:01:02 +0000 2013',
 u'entities': {u'hashtags': [{u'indices': [49, 60], u'text': u'Camlistore'}],
  u'symbols': [],
  u'urls': [{u'display_url': u'camlipy.readthedocs.org/en/latest/',
    u'expanded_url': u'https://camlipy.readthedocs.org/en/latest/',
    u'indices': [61, 84],
    u'url': u'https://t.co/9GNCYTvvCu'}],
  u'user_mentions': []},
 u'favorite_count': 0,
 u'favorited': False,
 u'geo': None,
 u'id': 369579788119199746,
 u'id_str': u'369579788119199746',
 u'in_reply_to_screen_name': None,
 u'in_reply_to_status_id': None,
 u'in_reply_to_status_id_str': None,
 u'in_reply_to_user_id': None,
 u'in_reply_to_user_id_str': None,
 u'lang': u'en',
 u'place': None,
 u'possibly_sensitive': False,
 u'retweet_count': 0,
 u'retweeted': False,
 u'source': u'web',
 u'text': u'I just released Camlipy 0.1.1, Python client for #Camlistore https://t.co/9GNCYTvvCu',
 u'truncated': False,
 u'user': {u'contributors_enabled': False,
  u'created_at': u'Tue Nov 09 22:56:48 +0000 2010',
  u'default_profile': False,
  u'default_profile_image': False,
  u'description': u'Python developer, paranoid about backups (http://t.co/HS7Z0kW9Iy and http://t.co/0Zwvyje93x)',
  u'entities': {u'description': {u'urls': [{u'display_url': u'docs.bakthat.io',
      u'expanded_url': u'http://docs.bakthat.io',
      u'indices': [42, 64],
      u'url': u'http://t.co/HS7Z0kW9Iy'},
     {u'display_url': u'bakserver.bakthat.io',
      u'expanded_url': u'http://bakserver.bakthat.io',
      u'indices': [69, 91],
      u'url': u'http://t.co/0Zwvyje93x'}]},
   u'url': {u'urls': [{u'display_url': u'thomassileo.com',
      u'expanded_url': u'http://thomassileo.com',
      u'indices': [0, 22],
      u'url': u'http://t.co/9g1uECbDWb'}]}},
  u'favourites_count': 26,
  u'follow_request_sent': False,
  u'followers_count': 132,
  u'following': False,
  u'friends_count': 348,
  u'geo_enabled': False,
  u'id': 213842895,
  u'id_str': u'213842895',
  u'is_translator': False,
  u'lang': u'fr',
  u'listed_count': 5,
  u'location': u'France',
  u'name': u'Thomas Sileo',
  u'notifications': False,
  u'profile_background_color': u'FFFFFF',
  u'profile_background_image_url': u'http://a0.twimg.com/profile_background_images/309927553/nvblackplaid_twitter.br.jpg',
  u'profile_background_image_url_https': u'https://si0.twimg.com/profile_background_images/309927553/nvblackplaid_twitter.br.jpg',
  u'profile_background_tile': False,
  u'profile_image_url': u'http://a0.twimg.com/profile_images/2383014688/mflvmo7ahhlvfgy945c6_normal.jpeg',
  u'profile_image_url_https': u'https://si0.twimg.com/profile_images/2383014688/mflvmo7ahhlvfgy945c6_normal.jpeg',
  u'profile_link_color': u'262626',
  u'profile_sidebar_border_color': u'000000',
  u'profile_sidebar_fill_color': u'545454',
  u'profile_text_color': u'9C9C9C',
  u'profile_use_background_image': True,
  u'protected': False,
  u'screen_name': u'trucsdedev',
  u'statuses_count': 114,
  u'time_zone': u'Paris',
  u'url': u'http://t.co/9g1uECbDWb',
  u'utc_offset': 7200,
  u'verified': False}}

Conclusion

Don't hesitate to let me know if you have any questions/suggestions !

Thomas Sileo

Create Release and Upload Build Artifact on GitHub from the Command Line

Your feedback

Setting Up SSL on AWS CloudFront Using your Own Domain

Creating the certificate

Upload to AWS IAM

Create/update your CloudFront distribution

Your feedback

Creating a WordPress Plugin to Integrate an External Script

The directory structure

Hooking the admin dashboard

Including the script

Installing the widget

Source Code

Useful Links

Bonus: Start a WordPress sandbox with Docker

Bonus: The Makefile

Your feedback

Building an embeddable Javascript widget (third-party javascript)

RequireJS crash course

Handling CSS

Embedding CSS

Handling configuration

Communicating with external server

Loading the code

A basic widget

Interesting links/projects

Publish a Message to Multiple Channels with Redis Pub/Sub and Lua Scripting

Your feedback

Parsing the Bitcoin block chain with Go language using Gocoin

The code

Feedback

Tracking Changes in Directories with Python

Files and subdir

Create an index

Last modified time

Sha256 hash

Final index

Computing changes

Dirtools

Your feedback

Using Yeoman with Flask

Flask

Grunt watch

Live reloading support

Share Flask config with your Javascript app

Introducing Flask-Yeoman

Feedback

Monitor Transactions on Bitcoin Addresses with Python and TxWatcher

Feedback

Playing with Python and cgminer RPC API

Enable cgminer API

Accessing the API with Python

Cgminer API wrapper

Pycgminer

Feedback

Rigs monitoring: a self-hosted monitoring dashboard for your mining rigs (Bitcoin mining) written in Python

Using Yeoman with a Custom Development Server

Server side

Live reloading support

Your feedback

Backing up your tweets to Camlistore with Python and Camlipy

Requirements

Twitter API

Getting started with Camlistore and Camlipy

The final script

Accessing tweets

Conclusion

Camlipy: Unofficial Python client for Camlistore

Bakthat 0.5.0 released with OpenStack Swift support and BakManager integration

OpenStack Swift Support

BakManager integration

Your feedback

How to convert seconds to human readable interval back and forth with Python

Backing up MongoDB to Amazon Glacier/S3 with Python using sh and bakthat

Features

Requirements

The script

Restore a backup

Miscellaneous