Sunday, January 13, 2013

shell.function to download all Repositories from a GitHub account

a shell utility to download all repositories managed under any GitHub account in the current directory...
if the current directory has any of the repositories already cloned, it pulls the latest

usage:   $ clone-github github
would download all the repositories managed under 'github' account available at https://github.com/github

#!/bin/bash
# Usage Example: $ clone-github github_id
# would clone all the git repositories managed under github_id at current path
# save it as /etc/profiles/a.github.sh
##
clone-github(){
GITHUB_ID=$1
GITHUB_REPO_URI="https://github.com/"$GITHUB_ID"?tab=repositories"
repos=`curl -skL $GITHUB_REPO_URI | grep 'name codeRepository' | sed 's/.*href=\"//' | sed 's/".*//'`
for line in `echo $repos | xargs -L1`;
do
if [[ -z $line ]]; then
next
fi
repo_git='git://github.com'$line'.git'
repo_dir=$(basename $line)
if [[ -d "${repo_dir}" ]]; then
echo 'Fetching master latest pull for: '$repo_git
pushd "${repo_dir}" ; git pull ; popd
else
echo "Cloning... "$repo_git
git clone $repo_git
fi
done
}
view raw a.github.sh hosted with ❤ by GitHub
######## _library functions
######## can infer from 'setup-chruby' and 'setup-fzf' at end on how to utilize these
download-release-from-github(){
local REPO_PATH="$1"
local REPO_VERSION="$2"
local TAR_GZ="$3"
local GITHUB_DDL_HEADERS=$(curl -Iks "https://github.com/${REPO_PATH}/releases/download/${REPO_VERSION}/${TAR_GZ}")
local URLTO_DDL=$(echo "$GITHUB_DDL_HEADERS" | grep '^Location: ' | awk '{print $2}' | sed 's/[[:space:]]*$//g')
local HEADER_GITHUB_REQUEST_ID=$(echo "$GITHUB_DDL_HEADERS" | grep '^X-GitHub-Request-Id: ' | awk '{print $2}' | sed 's/[[:space:]]*$//g')
local HEADER_REQUEST_ID=$(echo "$GITHUB_DDL_HEADERS" | grep '^X-Request-Id: ' | awk '{print $2}' | sed 's/[[:space:]]*$//g')
curl -H "X-GitHub-Request-Id: ${HEADER_GITHUB_REQUEST_ID}" -H "X-Request-Id: ${HEADER_REQUEST_ID}" -Lk -o "${TAR_GZ}" "${URLTO_DDL}"
}
download-archive-from-github(){
local REPO_PATH="$1"
local REPO_VERSION="$2"
local TAR_GZ="$3"
}
setup-from-github(){
local REPO_PATH="$1"
local REPO_VERSION="$2"
local TAR_GZ="$3"
local SETUP_CMD="$4"
local RELEASE_OR_ARCHIVE=${5:-release}
set -ex
pushd /tmp
if [[ "${RELEASE_OR_ARCHIVE}" == "release" ]]; then
download-release-from-github "${REPO_PATH}" "${REPO_VERSION}" "${TAR_GZ}"
elif [[ "${RELEASE_OR_ARCHIVE}" == "archive" ]]; then
local TARBALL_URI="https://github.com/${REPO_PATH}/archive/${TAR_GZ}"
wget -c -O "${TAR_GZ}" "${TARBALL_URI}"
else
echo "[error] unidentified github download type"
fi
tar zxvf "${TAR_GZ}"
eval "${SETUP_CMD}"
rm "${TAR_GZ}"
popd
set +ex
}
###################### _examples :: release uploads
setup-fzf(){
local REPO_PATH="junegunn/fzf-bin"
local REPO_VERSION="0.17.5"
local TAR_GZ="fzf-0.17.5-linux_amd64.tgz"
local SETUP_CMD="mv ./fzf ${HOME_BINDIR}/fzf"
[[ $(fzf --version &>/dev/null ; echo $?) -eq 0 ]] && \
echo "* fzf is already setup" && \
return 0
setup-from-github "${REPO_PATH}" "${REPO_VERSION}" "${TAR_GZ}" "${SETUP_CMD}"
}
###################### _examples :: release source code archives
setup-chruby(){
local CHRUBY_VERSION="0.3.9"
local CHRUBY_DIR="chruby-${CHRUBY_VERSION}"
local FILE_TO_SOURCE="/usr/local/share/chruby/chruby.sh"
local REPO_PATH="postmodern/chruby"
local REPO_VERSION="v${CHRUBY_VERSION}"
local TAR_GZ="${REPO_VERSION}.tar.gz"
local SETUP_CMD="pushd \"${CHRUBY_DIR}\" ; sudo ./scripts/setup.sh ; popd ; rm -rf \"${CHRUBY_DIR}\""
[[ -f "${FILE_TO_SOURCE}" ]] && \
echo "* chruby is already setup" && \
return 0
setup-from-github "${REPO_PATH}" "${REPO_VERSION}" "${TAR_GZ}" "${SETUP_CMD}" "archive"
source "${FILE_TO_SOURCE}"
}

Thursday, November 29, 2012

shell.function to download all files from any GITHUB GIST URL

a shell utility to download all files part of a GIST url at provided download path

usage e.g. $ ddl-gist https://gist.github.com/4165864 /etc/profile.d/

howTo install
$ curl -L -o /etc/profile.d/ddl-gist.sh https://gist.github.com/4165864

actually, if you had this utlity... the usage example would install the utlity... chicken-egg

# Usage Example: $ ddl-gist 'https://gist.github.com/4137843' ~/Downloads/gists
# save the gist files at that URL in ~/Downloads/gists
##
ddl_gist(){
if [ $# -ne 2 ];
then
echo 'Failed. Syntax: $> ddl-gist GITHUB_GIST_URL DOWNLOAD_PATH'
return
fi
gist_url=$1
download_path=$2
echo '[*] Getting all GIST File URLs from '$gist_url
gists=`curl -ksL -H 'User-Agent: Mozilla/5.0' $gist_url | grep '<a\ .*href=".*/raw/' | sed 's/.*a\ .*href="//' | sed 's/".*//'`
echo '[*] Downloading all files'
for lines in `echo $gists | xargs -L1`;
do
if [ ! -z $lines ];
then
echo $lines
gistfile=`echo $lines | sed 's/.*\///'`
save_as=$download_path"/"$gistfile
echo "Downloading URL: https://gist.github.com"$lines
echo "to "$save_as"....."
wget -c -O $save_as "https://gist.github.com"$lines
fi
done
}
view raw gistfile1.sh hosted with ❤ by GitHub

Tuesday, November 27, 2012

auto-download all media from RSS - DefCon20 Talk Videos

a shell utility code for download all media present as enclosures in any RSS

eg. usage to download all slides video of DefCon20
$ ddl-rss-media https://www.defcon.org/podcast/defcon-20-slides.rss

to install it
$ sudo curl -L -o /etc/profile.d/ddl-rss-media.sh https://gist.github.com/4156364#file_gistfile1.sh

#####
## $ ddl-rss-media https://www.defcon.org/podcast/defcon-20-slides.rss
#####
ddl-rss-media(){
# ddl-rss-media RSS_LINK {would download all media enclosed at current dir}
enclosures=`curl -k -s -L $@ | cat | grep enclosure | sed 's/.*enclosure\s*url="//' | sed 's/".*//'`
for url in `echo $enclosures | xargs -L1`;
do
if [ ! -z $url ];
then
filename=`echo $url | sed 's/?.*//' | sed 's/.*\///'`
echo "Downloading $filename..."
wget -c -O $filename $url
fi
done
}
view raw gistfile1.sh hosted with ❤ by GitHub

auto-download all videos of any Event at Confreaks.com

Auto-Download all smallest size Videos of any provided Event in Confreaks.com

to install it either run
$ sudo curl -L -o /etc/profile.d/ddl-confreaks.sh https://gist.github.com/raw/4137843/e6c6c93ef1c4cbf4cb735de821b74fcec467c170/gistfile1.sh

or copy+paste the shell function below in your shell-load-conf file.

suppose the event resides at URL:
http://confreaks.com/events/rubyconf2012

and you just need to run
$ ddl-confreaks rubyconf2012 ~/Downloads

and it will download all Talk Videos from Ruby Conference 2012 at your ~/Downloads directory...

would download all small Videos found at all talks under that Event Page

# Usage Example: $ ddl-confreaks rubyconf2012 ~/Downloads/
# save it as /etc/profiles/ddl.confreaks.sh
##
## currently it checks for lowest resolution video mostly {640x360} and downloads it
function ddl-confreaks(){
if [ $# -ne 2 ];
then
echo 'Failed. Syntax: $> ddl-confreaks EVENT_NAME DOWNLOAD_PATH'
return
fi
event=$1
download_path=$2
event_url="http://confreaks.com/events/$event"
echo '[*] Getting all talk urls from the Event Page'
talk_urls=`curl -s $event_url | grep --color -A1 "title" | grep href | sed 's/.*href="/http:\/\/confreaks\.com/' | sed 's/".*//'`
echo '[*] Getting all MP4 Video URLs from the Talk Pages'
for lines in `echo $talk_urls | xargs -L1`;
do
xmp4_url=`echo $lines | xargs curl -s | grep 'application/x-mp4' | tail -1 | sed 's/.*href="//' | sed 's/".*//'`
if [ ! -z $xmp4_url ];
then
echo "Scanned: "$lines
mp4file=`echo $xmp4_url | sed 's/.*\///' | sed 's/\?.*//'`
save_as=$download_path"/"$mp4file
echo "Downloading URL: "$xmp4_url
echo "to "$save_as"....."
wget -c -O $save_as $xmp4_url
fi
done
}
view raw gistfile1.sh hosted with ❤ by GitHub

Tuesday, October 9, 2012

why re-invent the wheel... Music Analogy

have been a curious kid...
used to spend hours finding new mathematical formulae ending up knowing it will be taught in higher class...

have been hearing a preaching since childhood, don't try to re-invent the wheel... but as any other kid never listened
and still don't.....

it's like Religion asking people not to find out how man came into existence as the Book already told...
and it's also like don't improve over it as there is a way to do it by someone very respected.....

now why I suddenly decided to blabber on such a topic where everyone already have there opinion and so.....
again, just trying to re-invent the wheel (else it would remain the multi-gonal irregular shaped)

so, here is a new analogy for people believing in it or not...

Music.....
no no, I wouldn't discuss about different Genres evolving through centuries... that will be very deep and philosophical..... and also not particularly my area of expertise

would just give some example of same (urdu) poetry sung (not re-mixed) by different legendary (Indian/Pak) singers... and not describe anything as for people capable of forming there own opinion, this much is enough... for others, I don't care

Poetry (Ghazal by Mirza Ghalib) : Har Ek Baat Pe Kehte Ho Tum Ke 'Tu Kya Hai?'
#ing is 7,6,5,4,3,2,1 to give a contemporary, old and back flow


(7.) Papon, KarshKale, MidivalPunditz:  http://www.youtube.com/watch?v=7bPU07RQuYA



(1.) by Kundan Lal Saigalhttp://www.youtube.com/watch?v=NkQ0poMB0as


(2.) by Lata Mangeshkarhttp://www.youtube.com/watch?v=8zFw7YMJUSk


(3.) by Ghulam Alihttp://www.youtube.com/watch?v=uvcrWgjLguQ


(4.) by Aabida Parveenhttp://www.youtube.com/watch?v=bi6tSZFIf2o


(5.) by Jagjit Singhhttp://www.youtube.com/watch?v=DinEYZvLuxk