Restrict user command using SUDO in Linux

Linux Security Police : Sudo

In Linux world we can restrict a user to run a specific command that need to be root privileges. Just use sudo and give sudo permission to user. But wait, that will give a super user privileges to the given user. They can do any Root user do, reboot, shutdown, rm -Rf /*  . Oh my, so what can we do.

In sudo still we can permit a user to run a specific command as a root level user, without giving all the root access privileges. In the Sudo documentation, it stated there but need more understanding as i do when go to the documentation.

So here my aim to just target the specific need to restrict a user run a specific command that need root privileges without giving all the root level privileges. So lets start with the use case.

I need a user  account that can execute a program that can start a service. User cannot restart other services than we specified. Let say the service is vpnconnect.sh , located in /usr/sbin . user just run sudo vpnconnect.sh restart to restart the service.

So we use visudo to edit the sudo files for safety and auto checking for errors when saving the file. Here are the step :

1. Add a command alias in the sudo file using visudo.
   
   Cmnd_Alias VPNC = /usr/sbin/vpnconnect.sh
   
2. Add the usergroup to allow run the command
   
   %support   ALL = (ALL)  VPNC
   
3. Create a group called support
   
   #groupadd support
   
4. Create a username and add it to the group.
   
   #useradd superman -g support
   #passwd superman
   
5. Thats it!

So now you have create a user with username superman. try login using SSH and then execute the command.

#sudo /usr/sbin/vpnconnect.sh restart

It will prompt for password to able to run it. If you try it with other user which not yet registered on the support group, it will fail.

So thats all. Normal user will not able to run command that need superuser account level like reboot, restart , restart and stop a service. 

Hope this helps.

Read More

Rest API with Django REST framework

When working with REST Api in your own application, you want to create a REST api which is follow the web standard of REST. In python , especially Django Worl, you can use Django REST framework. It support the REST Method , GET, POST, PUT, DELETE and handle it for you in the back ground.

So with this blog post, i want to have a post about my experience on working with Django REST framework. I never use it until my requirement for application goes to Mobile app, which surely the front end can be build using any hybrid method using javascript library/frameworks available, and in the back end, Python + Django still my super hero.

OK lets start, the Views in Django will be the same with Views in Django REST framework. And also the model will still be the same. We will reuse the model we already have. So the flow of creating a REST api in Django REST framework are :

1. Create Model
2. Create Serializer for the model
3. Create view for the model
4. Assign to the url.py of django

With this you will get the API browseable UI with only the above effort for you to test the API.

Step1. The models.py

from django.db import models

class Sms(models.Model):
    created = models.DateTimeField(auto_now_add=True)
    message = models.CharField(max_length=100, blank=True, default='')
    phone = models.TextField()
   
    class Meta:
        ordering = ('created',)

Step 2. the serializers.py

from rest_framework import serializers
from sms.models import Sms


class SmsSerializer(serializers.Serializer):

      model = Sms
        fields = ('id','created','message','phone')

Step 3. The view.py

from rest_framework import status
from rest_framework.decorators import api_view
from rest_framework.response import Response
from sms.models import Sms
from sms.serializers import SmsSerializer


@api_view(['GET', 'POST'])
def sms_list(request):
    """
    List all sms, or create a new sms
    """
    if request.method == 'GET':
        sms = Sms.objects.all()
        serializer = SmsSerializer(sms, many=True)
        return Response(serializer.data)

    elif request.method == 'POST':
        serializer = SmsSerializer(data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data, status=status.HTTP_201_CREATED)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)


@api_view(['GET', 'PUT', 'DELETE'])
def sms_detail(request, pk):
    """
    Retrieve, update or delete a sms instance.
    """
    try:
        sms = Sms.objects.get(pk=pk)
    except Sms.DoesNotExist:
        return Response(status=status.HTTP_404_NOT_FOUND)

    if request.method == 'GET':
        serializer = SmsSerializer(sms)
        return Response(serializer.data)

    elif request.method == 'PUT':
        serializer = SmsSerializer(sms, data=request.data)
        if serializer.is_valid():
            serializer.save()
            return Response(serializer.data)
        return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

    elif request.method == 'DELETE':
        sms.delete()
        return Response(status=status.HTTP_204_NO_CONTENT)

step 4. The url.py :

from django.conf.urls import url
from rest_framework.urlpatterns import format_suffix_patterns
from sms import views

urlpatterns = [
    url(r'^sms/$', views.sms_list),
    url(r'^sms/(?P[0-9]+)$', views.sms_detail),
]

urlpatterns = format_suffix_patterns(urlpatterns)

So the above step is how to create an api with your way, and full control on the views method you do before return the data to the rest api users.

There are another way to reduce your typing with Django REST framework, and i will post in on another blogpost.

So here it is, Django REST frameork.

Read More

updgrade Django-1.7.11 to 1.8.13

After upgrade my django to 1.7.11 , now the next version is 1.8.13.

So as usual i create a new virtualenv, with django-1.8.13 as the requirement, install the env with pip install.

The application affected which i use some are documented here.

1. Error on django-compressor-1.6 library.
   I try upgrade to django-compressor-2.0 and it fixed.
2. Django complain about south database module 'south.db.postgresql_psycopg2'. The warning tell to just remove south or using other supported database in south on SOUTH_DATABASE_ADAPTER settings.
   As Django have the build in migration, i will remove south package from my installation settings, and also remove the package in my virtualenv, so the warning message go away.
3. Warning on appconf modules. we use django-appconf-0.6 which required by django-compressor.
   The latest version is 1.0.1 , and after upgrade the warning is gone.
4. Django nose errors when running test. Need upgrade from django-nose-1.2 to django-nose-1.4.3
5. Test using py.test got error because django test need using database access.
   Upgrade the pytest-django-2.7.0 to pytest-django-2.9.0 will make test work again.

Thats all what i found when upgrading my django-1.7.11 to django-1.8.13 latest version.

Next will try to django-1.9.7 the latest version as now.

Hope this post help you Djangoers.

Read More

Django Test Improvement Speed

Testing in software development is a tedious task, and some programmers skip it. But many start to do the right thing from the beginning, Test Driven Development (TDD) movement make software developer more aware on delivering properly tested battle proven software. I also using TDD with my Django development.

One of factor when your test getting more and more in quantity, the test running slow as the test increased as your software getting more complex. I notice that my test is 135 cases, and need to wait 25 seconds to finish all the test. The test way run using py.test and using Django test also. Of course the test using SQLite on memory database to speed up the test. Also Pytest give the test speed an increase.

Then i read on documentation that for testing, Django still using default setup on other thing in the framework which can be removed in testing environment, like no middleware, no add on or things that slowing down. 1 i try are change the password hashed in django. The password hasher used by Django authentication was PBKDF2 algorith with a SHA256 hash, a password strecthing mechanism recomended by NIST. This should be sufficient for most users. It's quite secure, requiring massive amounts of computing time to break. This was from Django documentation. Sure it is save, and sure need more time and processing power to encrypt it. So can we change it in testing environment, to a faster password hasher. Especially when in testing, we have a lot of login and logout session, and it will make the testing longer.

You see in the documentation also state that django have other hasher, one is md5 which is common used for hasing a file and fast enough. Try to use that in the test environment and change the password hasher settings and see the improvement gains. Of course you need to have a test.py settings file used for your testing environment database and other config. Add this entries for hashing using md5.

#Use faster password hasher for testing
PASSWORD_HASHERS = (
    'django.contrib.auth.hashers.MD5PasswordHasher',
)

I try run my test, the improvement was really fast. Here are the screenshoot of my test without the md5 password hasher which using default hasher.

Raw Django Test

Then below the result when using md5 password hasher.

Optimized Django Test

So the improvement was 4 times the speed on testing. Total of 135 test case. I am happy to write my test again as much as possible to increase the quality of my code. Make testing a habit. No more worries when change something on the code will break other part of the code. Test suite will come to the rescue.

Happy Testing and Happy Coding. Testing and Coding should be fun with python.

Read More

Upgrading Django-1-6 to 1-7 part 2

This is part 2 of upgrading Django-1.6 to Django-1.7 journey. For part 1 you can go Here.

First error was my application using a module that was deprecated since Django-1.6 which already give me warning to fix it, but i ignore it. And i remember that, so i remove it and update with other lib.
The lib i use was django.utils.simplejson.

from django.utils import simplejson 

I change it to using json lib.

import json

Next is the testing give a warning that i am using settings  from django-1.5 which should be updated. I found out it was the test runner should be stated in the settings.py. So addition of the settings was added :

TEST_RUNNER = 'django.test.runner.DiscoverRunner'

Then test suite not give any warning message again, and the test all success.

Next is the django-debug-toolbar, i was using old version which are 1.0.1, and need to upgrade to django-debug-toolbar-1.4 because the import is giving error when using old version of django-debug-toolbar.

Next is the Django-compressor , i was using django-compressor-1.4 , and need to upgrade todjango-compressor-1.6.

And also change in HttpResponse , if you use mimetype , you should change it to content_type.

    return HttpResponse(json.dumps(dataxx), mimetype='application/json')

Update to this :

    return HttpResponse(json.dumps(dataxx), content_type='application/json')

And last one is the migration which already using south. If the app is newly developed and no db migration, no migration at all will give no problem. What if you have already migration in south running and deployed on the production, now you start develop using new django version, the migration will be need to use django build in migration. Well as in Django-1.7 migration already build in and use south,as the author said it will support on django-1.7.
Below is from south documentation.

This is the last major release of South, as migrations have been rolled into Django itself as part of the 1.7 release, and my efforts are now concentrated there. South will continue to receive security updates but no feature updates will be provided.

So for existing migration files, move the migrations folder to south_migrations, as django will use migrations folder for it migration. Then you need upgrade to south-1.0.2 which will first look in south_migrations directory, and fallback to migrations directory if not found.

And of course, south will not run on django-1.7.11. How we going to do with previous south migration created? we can still use django-1.6 with virtualenv, and run the south migration from there. But it is not recommended as in production will be to hard to maintain 2 virtualenv and also 2 migration in the long run. So when upgrade finish, better make django migration hold all the migration.

Next is in django model forms. You must specify the field you will include in the form creation. Django-1.7 give warning that it will removed in Django-1.8. So better clean up before next update will give you complete errors.

So hope my journey here will give other Djangonouts a heads up on the update bump you might encountered.

Read More

Upgrading Django-1-6 to 1-7 part 1

Currently Django already in version 1.9 the latest. And my Django app developed on Django version 1.6 since my last update. As the support and update will not delivered to version 1.6, better to upgrade it to the next version, which is 1.7 .

In upgrading a Python web framework like Django, the path not easy especially if you already build and using many library. Some will support the new version, some just not available for the new version of Django. As 1.7 is long enough, i will try my upgrade with virtualenv just like my previous upgrade from 1.5 to 1.6 in here.

So why need to upgrade. Actually, the why is have different answers on each case. For me, i want to have the update and feature Django new version offers, but provide stable framework for my development and app. Because so many component already in use and to make sure the app not break is the first priority. Some library i use were django-compressor, django-debug-toolbar, pytest for testing, south for database migration.

Some of the library like south migration, in django-1.7 already have the migration in core, so will need to remove south and use default migration shipped with django.

I will make this blog as an upgrade process log which i run step by step. Some notes are currently i am upgrading from Django-1.6.5 . If you still running Django-1.5.x you need upgrade it to django-1.6.x first.
As what best practice to do on every upgrade, one should read about the change logs in the new version. The new change in django-1.7 was :

1. Migration in core , which can replace south-migration lib if you use it.
2. Some deprecated lib like django.utils.simplejson. You can use json in new django-1.7
3. Test runner setup that changes
4. Support Python > 1.7

For details release notes can go to Django-1.7 Release notes

Let's start the migration path on new Django-1.7.11 from Django-1.6. First of all, we will using virtualenv in python for the upgrade. First we have the running old django in virtualenv, we need create new virtualenv and install the new Django-1.7.11

# cd /home/masterman/
# makevirtualenv dj-1.7.11
# workon dj-1.7.11

Then we can now install all the package we need for the webapp. I assume you will have requirements files to be feed to pip for installation. If you have the django requirement, change the version of django to Django==1.7.11

Then run the installation using pip :
# pip install -r requirements.txt

It will install all the requirement lib of the project you have, including django-1.7.11

I assume you project still intact, and in my case is /home/masterman/super-ecommerce/
Because i have test build in my app, and no changes, i can test it using django-1.7.11.

#py.test

This will test all my test file and give error if encountered. Thats why we need a test in every application we build so this things help us in migration or updates. Test is worth the effort.

And then voila, if there is errors you will spot where the errors are. And for me, i found some errors regarding the application library i use.

Continue on next part >>>

Read More

Postgresql Database basic Tune up

Postgresql is a robust database. it can server thousand of request per minute, if tune properly.

Some basic config you can do for tune the database are documented below.

Memory use :

shared_buffers = 2GB
# RAM/4 up to 8 GB

work_mem = 32MB
#Non shared memory use for sort , etc
# 8 MB to 32 Mb: web
# 128MB to 1 GB: reporting
# limit : Ram/(max_connection/2)

effetive_cache_sie = 6GB
# 3/4 of RAM

wal_buffers = 64Mb
# just set it

maintenance_work_mem = 512MB
# RAM/32
# more for reporting , use by analyze and autovacuum

checkpoint_segments = 64
# make WAL bigger
# space / 32 MB

checkpoint_completion_target = 0.9

stats_temp_directory = '/mnt/ramdisk'
#help with latency

random_page_cost = 1.5
# for AWS, SSD , decision for using index / io

effective_io_concurrency = 4
#for AWS, SSD, RAID

Logging Part , can use with pgBadger to read all the logs and view statistics :

log_connections = on
log_disconnections = on
log_temp_files = 1kB
log_lock_waits = on
log_checkpoints = on
log_min_duration_statement = 0

What we can do for optimize postgresql :

- Do less querying
- fix resoiurce-hungry requests
- get adequate hardware
- scale your infrastructure
- tune the config
- do caching on your application

source :
https://www.youtube.com/watch?v=dBeXS5aFLNc&list=PLE7tQUdRKcyaRCK5zIQFW-5XcPZOE-y9t

Read More