Postgresql backup and restore

When doing some development, we need a fast backup and restore for database data in postgresql server.

The package provide an easy way to do the backup and restore in 1 command.

You only need the username and password + the database name to be working on.

Here are the backup command signature :

Backup : $pg_dump -U {user-name} {source_db} -f {dumpfile.sql}

Restore: $psql -U {user-name} -d {destination_db} -f {dumpfile.sql}

So this command is straighforward. I dump a 1000 records in 5 seconds.

Also note that the privilege and db owner is follow in the dump file. You must prepare the exact username and database, with no table in it.
This is because the generated dump database not include a drop table command.


But how if we want to backup all database ? Of course we can do it to.

Backup all postgres databases :

We can backup all databases in postgres using pg_dumpall command .

To do the backup run this command :

$pg_dumpall > all.sql

We also can verify if all database is backed up using a grep command :

$grep "^[\]connect" all.sql
\connect blog
\connect facebook
\connect mytweet

What if we want to backup a specific postgres table ? hell yeah you can .

Backup a specific postgres table

The command are :

$pg_dump --table production -U acongbebo -f onlyproduction.sql

Postgresql is a powerfull database you can imagine. support for GIS database already included.

Read More

Postgresql DB Initialization in FreeBSD

Here are some command that used for administrating postgresql database.

The OS being use was FreeBSD9.1 with postgresql 8.3 

TO enable postgresql service add this in /etc/rc.conf :

postgresql_enable=”Yes”

postgresql_data=”/usr/local/pgsql/data”

postgresql_flags=”-w –s –m fast”

postgresql_initdb_flags=”—encoding=utf-8 –lc-collate=C”

postgresql_class=”default”

To install init db :

#/usr/local/etc/rc.d/postgresql initdb

We don’t have root user and a user in freebsd, so create one.

#su pgsql

#createuser dba    [set as
superuser]

Then change password :

#psql postgres dba

postgres=# ALTER
USER dba WITH PASSWORD ‘newpass’;

To create database :

#createdb mydb –o acong

[o] is owner of db

There we are. The database mydb ready to use and access from local host only.

Later on how to do backup and restore in Postgresql fast.

Read More

cacti time problem

Yesterday, I have a migration for may cacti NMS server from one Virtual Machine to other Virtual Machine. The problem is that the time error when the VM migrated.

This make the graph cannot be updated by the cacti pooler.

The time is shifted forward 3 years. If only 24 hours shift, we can just wait for the 24 hours time difference. But 3 years, its unsusable. So the solution is only delete all rrd data and crso bad, create from scratch.

This is so bad, as I don't have any backup. Last 1 year performance history data was gone.

This is bad with cacti, if only had a protection to stop the rra being updated by the pooler when time is shifting to far away.

I don't know if this can be happen with other monitoring tools like zabbix or zennos.

I like to use zabbix, but the setup to much and I have no time to try it out.

Lesson learned with CACTI :

1. Backup your RRA data once every week
2. Stop the pooler work when do some migration and time changing routine
3. Always make sure you have backup.

Hope you can learn from this experience also.

Read More

Upgrading Django 1.3 to 1.6 road blocks

So I try to upgrade my project to use Django 1.6 .

What I found is some setting that change, and to specific to note in the django Documentation. So I make notes on what have change and need to adjust in my django app configuration.

manage.py

The manage.py files. Now start from Django 1.4 there was a major change. One of the aim was to overcome the double import file in python.

The recommended manage.py are :

#!/usr/bin/env python
import os
import sys

if __name__ == "__main__":
  os.environ.setdefault("DJANGO_SETTINGS_MODULE", "{{ p_name }}.settings")
  from django.core.management import execute_from_command_line
  execute_from_command_line(sys.argv)

Here you change the {{ p_name }} to your project folder. And the manage.py should be move up one level of the project folder.

So the layout should be :

project

|--Blogs

   |-- urls.py

   |-- settings.py

   |-- models.py

   |-- views.py

|-- manage.py

urls.py

The urls.py import change. In Django1.3 we use :

from django.conf.urls.defaults import patterns, include, url

Now starting Django 1.4 there were no more defaults in urls. So we change to :

from django.conf.urls import patterns, include, url

To make it compatible with previous version of Django1.4 you can use try block :

try:
  from django.conf.urls.defaults import patterns, include, url # django1.3 support
except:
  from django.conf.urls import patterns, include, url

settings.py

If you implement the resusable app concept, your settings will have some application level configuration variable in the project settings.py which you can access from application via settings.import . But after Django1.4 all change. 

If your application need access configuration variable in settings.py , this what you should change:

from settings import RESULTS_PER_PAGE , SITE_NAME, SITE_DESCRIPTION

To new way :

from django.conf import settings
  RESULTS_PER_PAGE = settings.RESULTS_PER_PAGE
  SITE_NAME = settings.SITE_NAME
  SITE_DESCRIPTION = settings.SITE_DESCRIPTION

if you want to keep compatibility with < Django1.4 :

try:
  from settings import RESULTS_PER_PAGE , SITE_NAME, SITE_DESCRIPTION
except:
  from django.conf import settings
  RESULTS_PER_PAGE = settings.RESULTS_PER_PAGE
  SITE_NAME = settings.SITE_NAME
  SITE_DESCRIPTION = settings.SITE_DESCRIPTION

And another thing to change in settings.py in TEMPLATE_CONTEXT_PROCESSOR section :

# old
TEMPLATE_CONTEXT_PROCESSORS = ("django.core.context_processors.auth",
                              
)
# new
TEMPLATE_CONTEXT_PROCESSORS = ("django.contrib.auth.context_processors.auth",
                               
)

So the change I found out only :

1. Manage.py file location
   This change the whole config and directory layout of a project
2. URLS import in urls.py
   No more django.conf.urls.defaults , change to django.conf.urls
3. Import Settings Variable
   Need other way to import settings variable. What I learn that any reusable app should not depends on the project settings.This give a loose app and project relationship.

And the result, I bumped with more configuration that had to change, like the context processor utils?

I decide to roll back, and better use the new Django1.6 with new project anyway.

And maybe should upgrade 1.3 to 1.4 first.

Updates :

I try again and bumped with the context processor utils, the problem was the django.conf import. I use import settings for load all the default knob in my application, now i use :

from django.conf import settings  

Now my apps run in django 1.4 . Next will be upgrade to django 1.5 and I will posting my findings.

Read More

Django Version 1.6 release

Recently Django released Django 1.6 release in 6 November 2013. While I am still using Django 1.3 in my production server.

Looks so many improvement and site layout arrangement in newer version and make the temptation to try it out.

I have the options to upgrade to the newer version. But what makes me stop is there are to many change in the process. Maybe better to use the newer version with new project because the layout is changes to much.

In my deployment and development, I use the Reuse application methodology. So i hope the change is only in the project layout, but the app layout is still the same.

What I encounter in Django 1.3 lack of is, the sql batch insert. In one of my project, I need to update many rows at once, and the ORM doing update to database one at a time.

In Django 1.4 there is sql batch insert functionality which is good news.

And in Django 1.6 that I read from realese notes, the update are :

• SQL persistance connection
• Admin interface activated by default
• Change in Django Transaction Handling
• Discovery of test in any test module
• Support BinaryField in model field
• Model.save() algorithm changed
  This minimize the queries sent to sql server to 1 command only, before 2 sql command when save is called

For the Details you can see in the Release Notes of Django sites.

Read More

Links to research data

I made this list for my own purpose to get research and paper information i found in Internet.

Hope usefull for you who found this page :

http://ageconsearch.umn.edu

Read More

Battle of the Year 2013 The Movie

Today i just notice about a movie about dance sport competition in one of online video website and watch the movie trailer. It is interesting because in the movie, the story about the sport, the coach and the team.

This what make me interested to watch the movie at first place

" A good coach can take his team to championship "

But this 

"A great coach can get any team in any sports to the TOP"

This something that greatness produce, and i think many things i can learn from this movie and also it is entertaining to see a dance competition and the choreographer . It makes me fell some spirits to do more in daily life.

So just check out your self the movie trailer






Battle of the Year Trailer Chris Brown 2013 Movie - Official [HD]

This will be my video collection . Hope you enjoy the information.

Read More