var javascript_array =  {{=XML(response.json(array))}}

Above statement was working on web2py 2.9.11 and before because of behavioral bug in web2py, which got fixed in web2py 2.9.11. Now response.json()  sets content-type = application/json . So because of above line, on web2py >2.9.11 html code is displayed in browser instead of rendering it as html.

Correct way:

First convert python variable to json in controller and then pass it to view. And in view, use XML() to assign it to javascript variable so that it is not escaped

Controller

from gluon.serializers import json

def index():
    array = [1, 2, 3, 4, 5]
    dictionary = dict(id=1, name='foo')
    return dict(array=json(array), dictionary=json(dictionary))

View

<script>
    var js_array = {{=XML(array)}};
    var js_dictionary = {{=XML(dictionary)}};
</script>

Other solution is use ASSIGNJS html helper

Controller

def index():
    array = [1, 2, 3, 4, 5]
    dictionary = dict(id=1, name='foo')
    return dict(array=array, dictionary=dictionary)

View

<script>
    {{=ASSIGNJS(js_array = array)}}
    {{=ASSIGNJS(js_dictionary = dictionary)}}
</script>

The post How to assign python variable to Javascript variable in web2py appeared first on Gaurav Vichare.

For all the web2py applications, web2py admin url is same, i. e. www.example.com/admin or 127.0.0.1:8000/admin. One should prevent access to admin interface  to public or external users. Admin url should be changed from /admin to /some-other-url which is difficult to guess. admin is just another app like welcome or example app, so changing appliacation name will change url to it.

For example to change admin  to w2p-adm2in  do following steps

1. Rename admin folder with w2p-adm2in
On linux

cd web2py/applications
mv admin w2p-adm2in

On Windows

cd web2py/applications
rename admin w2p-adm2in

If you are in local environment, you can also do it using your file explorer

2. Fix broken links
Due to changes done in Step 1, links to admin app are broken. Links to admin app needs to be replaced with links to w2p-adm2in.

In app_name/controllers/appadmin.py
Update response.menu, change it to following

response.menu = [[T('design'), False, URL('w2p-adm2in', 'default', 'design',
                  args=[request.application])], [T('db'), False,
                  URL('index')], [T('state'), False,
                  URL('state')], [T('cache'), False,
                  URL('ccache')]]

3. Instead of admin app, check user is logged in to w2p-adm2in app
while accessing appadmin, by default controller app_name/controllers/appadmin.py checks whether user is logged in to admin app or not, if not then it redirects to admin app(admin app login page), now it should check whether user is logged in to w2p-adm2in app and if not it should redirect to w2p-adm2in. To acheive this, pass other_application='w2p-adm2in'  as parameter to method gluon.fileutils.check_credentials()

After step 2 and step 3, diff file of app_name/controllers/appadmin.py will look like following

if not (gluon.fileutils.check_credentials(request) or auth.has_membership(manager_role)):
         raise HTTP(403, "Not authorized")
     menu = False
-elif (request.application == 'admin' and not session.authorized) or \
- (request.application != 'admin' and not gluon.fileutils.check_credentials(request)):
-    redirect(URL('admin', 'default', 'index',
+elif (request.application == 'w2p-adm2in' and not session.authorized) or \
+ (request.application != 'w2p-adm2in' and not gluon.fileutils.check_credentials(request, other_application='w2p-adm2in')):
+    redirect(URL('w2p-adm2in', 'default', 'index',
                  vars=dict(send=URL(args=request.args, vars=request.vars))))

ignore_rw = True
response.view = 'appadmin.html'
if menu:
- response.menu = [[T('design'), False, URL('admin', 'default', 'design',
+ response.menu = [[T('design'), False, URL('w2p-adm2in', 'default', 'design',
                    args=[request.application])], [T('db'), False,
                    URL('index')], [T('state'), False,
                    URL('state')], [T('cache'), False,
                    URL('ccache')]]

4. Change links to error page (tickets)
By default links to error is /admin/default/ticket/[ticket_id], Now it should be /w2p-adm2in/default/ticket/[ticket_id]
So add/replace/append following lines to web2py/routes.py

error_message = '<html><body><h1>{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}s</h1></body></html>'
error_message_ticket = '<html><body><h1>Internal error</h1>Ticket issued: <a href="/w2p-adm2in/default/ticket/{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}(ticket)s" target="_blank">{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}(ticket)s</a></body></html>'

If routes.py file does not exist in web2py folder then create it containing above lines.

If you more changes that needs to be done, please comment below.

The post How to change admin app url in web2py? appeared first on Gaurav Vichare.

1. IS_IMAGE

It checks whether uploaded file is of type image and file is in any of supported image formats

Following statement Checks if uploaded file is either png or bmp

requires = IS_IMAGE(extensions=('png', 'bmp'))

2. IS_UPLOAD_FILENAME

It checks whether uploaded file meets the given rules for filename and extension.

You can provide regular expressions in filename and extension parameters.
If filename or extension doesn’t match the expression/criteria then it shows error in form.
You can provide your own error message using argument error_message

Example:

If you want to allow all files starting with word ‘log’ and having extension “txt”:

requires = IS_UPLOAD_FILENAME(filename='^log', extension='txt')

Like IS_IMAGE validator we can not pass list of valid extensions to IS_UPLOAD_FILENAME validator.
But we can achieve this using regular expression or by writing our own validator

1. Using regular expression:

requires=IS_UPLOAD_FILENAME(extension='^(pdf|docx|doc|xls)$')

The above expression will allow all the files having extension pdf, docx, doc or xls.

^ Matches the start of string

$ Matches end of the string

(pdf|docx|doc|xls) match any value from group pdf, docx, doc and xls

Read more about Regular Expression

2. Using custom validator:

Following validator checks whether uploaded file has extension from the valid extension list provided through argument extensions. It will also show error for empty file.

import os

class IS_ALLOWED_FILE_TYPE(object):
    """
    Checks if file uploaded through file input has valid extension

    Args:
        extensions: iterable containing allowed *lowercase* file extensions
        error_message:

    Examples:
        Check if uploaded file is pdf or docx:

        INPUT(_type='file', _name='name',
              requires=IS_ALLOWED_FILE_TYPE(extensions=['pdf', 'docx']))

        db.table.field.requires = IS_ALLOWED_FILE_TYPE(extensions=['pdf', 'docx'],
            error_message='File is not pdf or docx'))
    """
    def __init__(self, extensions=[], error_message="File type not supported"):
        self.extensions = extensions
        self.error_message = error_message

    def __call__(self, value):
        error = None
        try:
            filename = value.filename
        except:
            return (value, self.error_message)

        _, file_extension = os.path.splitext(filename)
        if file_extension.replace('.', '').lower() not in self.extensions:
            error = self.error_message
        return (value, error)

Usage:

db.define_table('person',
                Field('name'),
                Field('template', 'upload',
                      requires=IS_ALLOWED_FILE_TYPE(extensions=['pdf' 'docx', 'doc'])))

Advantage of custom validator is you can add and modify restriction as per your need

If you are facing any issues regarding above post please comment below

The post How to do file validation in web2py and write custom validator to validate files? appeared first on Gaurav Vichare.

example: www.example.com/default/custom_download/1

Consider following employee schema:

db.define_table('employee',
                Field('name', 'string', required=True),
                Field('email', string, required=True),
                Field('address', 'text', required=True),
                Field('resume', 'upload', required=True))

Then custom download controller will be:

def custom_download(): 
    employee_id = request.args(0) 
    emp_record = db(db.employee.id == employee_id).select().first()
    download_file = emp_record.resume

    # Name of file is table_name.field.XXXXX.ext, so retrieve original file name
    org_file_name, file_stream = db.employee.resume.retrieve(download_file)

    file_header = "attachment; filename=" + org_file_name
    response.headers['ContentType'] = "application/octet-stream"
    response.headers['Content-Disposition'] = file_header
    return response.stream(file_stream)

If you have any queries related to this code, please comment below.

The post How to write custom download controller in web2py? appeared first on Gaurav Vichare.

To install PyPDF2:

sudo pip install pypdf2

We are using encrypt function of PyPDF2.

encrypt(user_password, owner_password=None, use_128bit=True)

• user_password  – The “user password” allows opening and reading the PDF file with the restrictions .
• owner_password – The “owner password”  have no restrictions. By default, the owner password is the same as the user password.
• use_128bit  – Decides which encryption to use128bit or 40bit.

import PyPDF2
import os
import argparse


def set_password(input_file, user_pass, owner_pass):
    """
    Function creates new temporary pdf file with same content,
    assigns given password to pdf and rename it with original file.
    """
    # temporary output file with name same as input file but prepended
    # by "temp_", inside same direcory as input file.
    path, filename = os.path.split(input_file)
    output_file = os.path.join(path, "temp_" + filename)

    output = PyPDF2.PdfFileWriter()

    input_stream = PyPDF2.PdfFileReader(open(input_file, "rb"))

    for i in range(0, input_stream.getNumPages()):
        output.addPage(input_stream.getPage(i))

    outputStream = open(output_file, "wb")

    # Set user and owner password to pdf file
    output.encrypt(user_pass, owner_pass, use_128bit=True)
    output.write(outputStream)
    outputStream.close()

    # Rename temporary output file with original filename, this
    # will automatically delete temporary file
    os.rename(output_file, input_file)


def main():
    parser = argparse.ArgumentParser()
    parser.add_argument('-i', '--input_pdf', required=True,
                        help='Input pdf file')
    parser.add_argument('-p', '--user_password', required=True,
                        help='output CSV file')
    parser.add_argument('-o', '--owner_password', default=None,
                        help='Owner Password')
    args = parser.parse_args()
    set_password(args.input_pdf, args.user_password, args.owner_password)

if __name__ == "__main__":
    main()

If you have any queries, please comment!

The post How to password protect PDF file using Python? appeared first on Gaurav Vichare.

sudo easy_install -U memory_profiler

1. Profile function/script:
   Add following line in script to import memory profiler:
   

   from memory_profiler import profile

   
   Decorate the function you would like to profile with @profile
   Example:
   

   #test.py
   from memory_profiler import profile
   
   @profile
   def test_func():
       numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9]
       addition = 0
   
       for num in numbers:
           addition += num
   
       return addition
   
   if __name__ == '__main__':
       test_func()

   
   Run python script to get memory usage line by line.
   

   python test.py

   
   Another method to profile function is , decorate function with @profile and then run script using following command:
   

   python -m memory_profiler test.py

   
   2. Profile web2py application/(External scripts):
   Start webpy server using following command :
   

   mprof run python web2py.py -p 8000 -a g0987

   
   Now open application from browser and open page you want to profile. Then stop web2py server. To view result run command ‘mprof plot’. But to plot result you need package ‘matplotlib’.
   To install matplotlib, first install required packages for matplotlib:
   

   sudo apt-get build-dep python-matplotlib

   
   Now install matplotlib:
   

   sudo pip install matplotlib

   
   Now run following command to get memory profile graph (Memory Usage vs Time):
   

   mprof plot

   
   The available commands for mprof :
   • mprof run: running an executable, recording memory usage
   • mprof plot: plotting one the recorded memory usage (by default, the last one)
   • mprof list: listing all recorded memory usage files.
   • mprof clean: delete recorded memory usage files.
   • mprof rm : delete particular  memory usage file.

The post Memory profiling in Python using memory_profiler appeared first on Gaurav Vichare.

1.  Profile single function
2. Profile complete python script
3. How to read .prof file
4. Profile web2py application

To install cprofile on ubuntu use following command:

sudo pip install cprofilev

1) Profile single function

To profile function, import cProfile

import cProfile

Then call function using cProfile.run()

cProfile.run('test()', 'test.profile')

This will write profiling data in test.profile .

Example:

import cProfile

def test_func():
    numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9]
    addition = 0

    for num in numbers:
        addition += num

    return addition

if __name__ == '__main__':
    cProfile.run('test_func()', 'test.profile')

Later in this post we will see how to read .profile file

2) Profile python script

Run python script using following command :

python -m cProfile -o test.profile test.py

This will profile complete script and write profiling data in test.profile.

3) How to read .profile file?

Using pstats module:

.profile file is in binary format. So to examine data from .profile file use the interactive mode of the pstats module by running it as a script with the profile data file as argument.

python -m pstats test.profile

Now you will go in interactive profile statistics browser.

Welcome to the profile statistics browser.
test.profile{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}

Now use stats command to display stats

Welcome to the profile statistics browser.
test.profile{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97} stats

You can sort stats using following sort keys:
cumulative — cumulative time
module — file name
ncalls — call count
pcalls — primitive call count
file — file name
line — line number
name — function name
calls — call count
stdname — standard name
nfl — name/file/line
filename — file name
cumtime — cumulative time
time — internal time
tottime — internal time

For example:

Welcome to the profile statistics browser.
test.profile{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97} stats tottime

Above command will sort records using tottime ( internal time)

To show particular number of records use command “stats count”

For example:

Welcome to the profile statistics browser.
test.profile{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97} stats 10

This will show only 10 records.

Other method to read .profile file is using command “cprofilev -f test_func.profile”

cprofilev -f test_func.profile

Run above command in terminal and view stats in browser on http://127.0.0.1:4000

Advantage of this method is you can easily sort records ,just by one click on column name.

4. Profile web2py application

Create a empty text file in a directory and then pass this text file as argument to web2py. All profiling data will be stored in this file.
For example :
Create lcm.txt in ‘/home/gaurav/temp/lcm/” directory. Now run web2py server using following command with command line argument -F (profiler filename)

python web2py.py -p 8000 -a g0987 -F /home/gaurav/temp/lcm/lcm.txt

Note: For some web2py version , we have to provide directory using -f (not -F) option and data will be written in that directory in the form of .profile files. We can read these files using above mentioned methods.

Now open application from browser and open pages you want to profile.Then stop web2py server. This will write profiling data to lcm.txt. Data in text file will be in readable format , no need of pstats module to read data.

The post Time profiling in python using cProfile appeared first on Gaurav Vichare.

What is profiling?

• Profiling is program analysis that measures, for example, the space (memory) or time complexity of a program, the usage of particular instructions, or the frequency and duration of function calls.
• Profiling or performance measurement is main tool for code optimization.
• There are two types of profiling : Time profiling and Memory profiling
• We should also consider memory leaks to optimize code.

1) Date

Run script by running date command before and after script to find time when script started execution and when it completed.

date; python test.py; date

This prints time before running script and again print time when execution is complete. So we can find execution time by subtracting end time from start time.

2) time

Unix time command is helpful to figure out how much time command, shell script or python program takes for execution.
Usage:

time python test.py

output:

python test.py  0.02s user 0.02s system 15{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97} cpu 0.301 total

• User : CPU time spent in user-mode (outside the kernel)
• System : CPU time spent in the kernel
• CPU : Percent of CPU this job got
• Total : elapsed time, start to end

Why user + system ≠ total ?
total represents actual elapsed time, while user and sys values represent CPU execution time.

3) /usr/bin/time

/usr/bin/time gives more detailed output than ‘time‘.
Usage:

/usr/bin/time -v test.py

Output:

/usr/bin/time: cannot run pycrypto.py: No such file or directory
Command exited with non-zero status 127
	Command being timed: "pycrypto.py"
	User time (seconds): 0.00
	System time (seconds): 0.00
	Percent of CPU this job got: ?{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}
	Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.00
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 344
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 0
	Minor (reclaiming a frame) page faults: 72
	Voluntary context switches: 1
	Involuntary context switches: 0
	Swaps: 0
	File system inputs: 0
	File system outputs: 0
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 127

What is difference between usr/bin/time and time ?
time is built in unix command and usr/bin/time is GNU time

The post How to measure performance of code ? – Basic methods appeared first on Gaurav Vichare.

Input

t – number of test cases [t < 1000]
On each of next t lines given a integer N [-1000 <= N <= 1000]

Output

One integer equals to sum of all positive integers.

Score

Score equals to size of source code of your program except symbols with ASCII code <= 32.

Example

<b>Input:</b>
4
5
-5
6
-1
<b>Output:</b>
11

C Program :

#include<stdio.h>
int main()
{
   unsigned long sum=0;

   short n,a,i;
   scanf("{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}hu",&n);
   for(i=0;i<n;i++)
   {
       scanf("{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}hu",&a);
       if(a>0)
       sum+=a;
   }
printf("\n{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}lu",sum);
   return 0;
}

The post Size Contest (SPOJ Challenge problem 378) appeared first on Gaurav Vichare.

The most important part of a GSM network is so called Base Transceiver Station (BTS). These transceivers form the areas called cells (this term gave the name to the cellular phone) and every phone connects to the BTS with the strongest signal (in a little simplified view). Of course, BTSes need some attention and technicians need to check their function periodically.

ACM technicians faced a very interesting problem recently. Given a set of BTSes to visit, they needed to find the shortest path to visit all of the given points and return back to the central company building. Programmers have spent several months studying this problem but with no results. They were unable to find the solution fast enough. After a long time, one of the programmers found this problem in a conference article. Unfortunately, he found that the problem is so called “Travelling Salesman Problem” and it is very hard to solve. If we have N BTSes to be visited, we can visit them in any order, giving us N! possibilities to examine. The function expressing that number is called factorial and can be computed as a product 1.2.3.4….N. The number is very high even for a relatively small N.

The programmers understood they had no chance to solve the problem. But because they have already received the research grant from the government, they needed to continue with their studies and produce at least some results. So they started to study behaviour of the factorial function.

For example, they defined the function Z. For any positive integer N, Z(N) is the number of zeros at the end of the decimal form of number N!. They noticed that this function never decreases. If we have two numbersN₁<N₂, then Z(N₁) <= Z(N₂). It is because we can never “lose” any trailing zero by multiplying by any positive number. We can only get new and new zeros. The function Z is very interesting, so we need a computer program that can determine its value efficiently.

Input

There is a single positive integer T on the first line of input (equal to about 100000). It stands for the number of numbers to follow. Then there are T lines, each containing exactly one positive integer number N, 1 <= N<= 1000000000.

Output

For every number N, output a single line containing the single non-negative integer Z(N).

Example

Sample Input:

6
3
60
100
1024
23456
8735373
Sample Output:

0
14
24
253
5861
2183837

 Program:

#include<stdio.h>

int main()
{
  int t;
  long long n,sum;

  scanf("{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}d",&t);
  while(t--)
  {    sum=0;
      scanf("{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}lld",&n);
      while(n)
      {
          n=n/5;
          sum+=n;
       }
      printf("{b15f91eef52e7c6d40fd3fe45227e26332e4c02be027e4e72c6ecf7adacefb97}lld\n",sum);
   }
return 0;
}

The post Factorial (SPOJ problem 11) appeared first on Gaurav Vichare.