grooveek.metaClass.blog = {shareDiscoveries()}

Playing again with zip and Groovy MOP ( and spread operator)

2009-09-15T01:22:00.004+02:00

After I posted the zip and unzip snippet, I came accross an interesting question on gr8forums.org.
Someone told me it would be great if ths code could be applied to an array of files.

That's a good use case, isn't it ? So, I thought about it and saw immediately which groovy 'magic item' could be useful : the spread operator aka star-dot operator, mister *.
Better than I could explain myself, here's what's said about this little thing in Groovy's doc :

The Spread Operator is used to invoke an action on all items of an aggregate object.

Unfortunately, ZipOutputStream in Java doesn't permit to append something to a zip file. So, I'll show you an awful (yes, unzip the existing zip-append file-rezip IS awful) implementation of this functionnality. It's just for fun, not for production, after all.

First, you'll need a deleteDir closure, as java.io.File.delete() doesn't delete non-empty directories :

def deleteDir
deleteDir = { File f ->
    if (f?.exists()){
      f.listFiles().each{ sf ->
        sf.isDirectory()?deleteDir(sf):sf.delete()
      }
    }
    f?.delete()
}

With this wonderful snippet, you can replace the File.metaClass.zip block in this snippet by :

File.metaClass.zip = { String destination ->
    //cache the delegate (the File Object) as it will be modified
    //in the withStream closure
    def input = delegate
    if (destination == null){
        destination = input.canonicalPath + ".zip"
    }
    def dest = new File(destination)
    def append = false
    def tmp
    if (dest.exists()){
        tmp = new File(System.properties['java.io.tmpdir'] + File.separator + 'grooz')           
        tmp.mkdir()
        dest.unzip(tmp.canonicalPath)
        append = true
    }
    def result = new ZipOutputStream(new FileOutputStream(destination))
    try{
    result.withStream {zipOutStream->
        //recursively zip files
        if (append) {
            
                tmp.eachFile{
                    zip(zipOutStream,it,"")
                }
            
        }
        zip(zipOutStream,input,"")
    }
    }
    catch(all){
      all.printStackTrace()
    }
    finally{                        
      deleteDir(tmp)
    }    
}

This version is very easy to use. If you want to zip a bunch of files, then list them and use the spread operator like this :

// to zip all files in a directory
new File('/path/to/dir').listFiles()*.zip('allmyfiles.zip') 
//if you've got files instances
[file1,file2,dir1,dir2]*.zip('stuff.zip')

This code is not performance-friendly nor beauty-frienly and is provided as is. Every enhancement you can think of is welcome (yes, I know, it'll be difficult to improve, but you can do it ;-) )

See you soon with a totally new subject : joy of Java Classoading !

Adding Zipping and unzipping capabilities to Java.io.File via Groovy MOP

2009-09-11T01:00:00.003+02:00

this post has an update : Playing again with zip and Groovy MOP ( and spread operator)

[First technical post, and I don't know how to begin... Please be kind with me]
So, as I'm writing a Plugin System in Groovy, I needed to zip and unzip so-called 'Bundles' which contain config files, Plugin jars and dependencies.

I didn't want to make a classical FileUtils or BundleUtils class, and I decided to try Groovy MOP (Meta Object Protocol) to directly modify the java.io.File so I could write

new File('/path/to.file').zip(/path/to.zip)

First thing to do is adding capability when you need to. As an example, I decided to load the capability statically in a Class. You could either have a bootsrap class that loads your metaprogramming functionnalities before starting your App. This is your choice

static{
  bootstrapPluginSystem()
}

private static void bootstrapPluginSystem(){
   //zipping methods here
   ..................
   //unzipping methods here
   ..................
}

Zipping files is easy in Java and is performed via the ZipOutputStream class.

One of the grooviest groovy goodies is the ability to work seamlessly with streams without boilerplate resource closing code
Simply use the withStream method, and Groovy manage the resource for you. How great it is !!!

Groovy MOP is wonderful to add methods to Class at Runtime. The simple example I take here demonstrates the power of this technique

Zipping is quite easy :

//define zip closure first, then allocate it
 //it is needed to make the recursion work
 def zip
 zip = { ZipOutputStream zipOutStream,File f , String path->                        
    def name = (path.equals(""))?f.name:path + File.separator + f.name
    if(!f.isDirectory() ){                            
      def entry = new ZipEntry(name)
      zipOutStream.putNextEntry(entry)
      new FileInputStream(f).withStream { inStream ->
        def buffer = new byte[1024]
        def count
        while((count = inStream.read(buffer, 0, 1024)) != -1) {
          zipOutStream.write(buffer,0,count)
        }
      }
      zipOutStream.closeEntry()
    }
    else {
      //write the directory first, in order to allow empty directories
      def entry = new ZipEntry(name + File.separator)
      zipOutStream.putNextEntry(entry)
      zipOutStream.closeEntry()                            
      f.eachFile{
        //recurse
        zip(zipOutStream,it,name)
      }
    }
 }

 File.metaClass.zip = { String destination ->
    //cache the delegate (the File Object) as it will be modified 
    //in the withStream closure
    def input = delegate            
    def result = new ZipOutputStream(new FileOutputStream(destination))
    result.withStream {zipOutStream->   
      //recursively zip files             
      zip(zipOutStream,input,"")
    }
 }

Code is almost self explaining...
Unzipping comes with less efforts :

File.metaClass.unzip = { String dest ->
 //in metaclass added methods, 'delegate' is the object on which 
 //the method is called. Here it's the file to unzip
 def result = new ZipInputStream(new FileInputStream(delegate))
 def destFile = new File(dest)
 if(!destFile.exists()){
   destFile.mkdir();
 }
 result.withStream{
   def entry
   while(entry = result.nextEntry){
     if (!entry.isDirectory()){
       new File(dest + File.separator + entry.name).parentFile?.mkdirs()
       def output = new FileOutputStream(dest + File.separator 
                                         + entry.name)                        
       output.withStream{
         int len = 0;
         byte[] buffer = new byte[4096]
         while ((len = result.read(buffer)) > 0){
           output.write(buffer, 0, len);
         }
       }
    }
    else {
      new File(dest + File.separator + entry.name).mkdir()
    }
   }
 }
}

All that is not really thoroughly tested, but I managed to zip and unzip the grails project directory without problems. I can also unzip with Ark files zipped by these lines of code

Here are minimal tests :

import groovy.util.GroovyTestCase
/**
 *
 * @author grooveek
 */
class ZipTest extends GroovyTestCase{
 // load the static block to add zip capabilities
 static def bootstrap = new BootStrap()

 void testZipUnzip(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   def writer = new File('testfiles/ziptest.txt').newWriter()
   100.times{
     writer.writeLine("$it")
   }
   writer.close()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/ziptest.txt')
   fileToZip.zip('testfiles/toto.zip')
   def zipToUnzip = new File('testfiles/toto.zip')
   zipToUnzip.unzip('testfiles/ziptestdir')
 }

 void testZipUnzipWithDirectories(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   new File('testfiles/toto/titi').mkdirs()
   def writer = new File('testfiles/toto/titi/ziptest.txt').newWriter()
   100.times{
     writer.writeLine("$it")
   }
   writer.close()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/toto')
   fileToZip.zip('testfiles/toto2.zip')
   def zipToUnzip = new File('testfiles/toto2.zip')
   zipToUnzip.unzip('testfiles/ziptestdir2')
 }

 void testZipUnzipWithDirectoriesAndMultipleFiles(){
   def fi = new File('testfiles')
   if (!fi.exists()){
     fi.mkdir()
   }
   new File('testfiles/tata/titi').mkdirs()
   5.times{
     def writer = new File("testfiles/tata/titi/ziptest${it}.txt").newWriter()
     100.times{
       writer.writeLine("$it")
     }
     writer.close()
   }
   new File('testfiles/tata/titi/toto').mkdirs()
   5.times{
     def writer = new File("testfiles/tata/titi/toto/ziptest${it}.txt").newWriter()
     100.times{
       writer.writeLine("$it")
     }
     writer.close()
   }
   new File('testfiles/tata/titi/toto/tutu').mkdirs()
   //testing zipping/unzipping
   def fileToZip = new File('testfiles/tata')
   fileToZip.zip('testfiles/toto3.zip')
   def zipToUnzip = new File('testfiles/toto3.zip')
   zipToUnzip.unzip('testfiles/ziptestdir3')
}
        

 void testZipBigAndDeepDirectory(){
   def fileToZip = new File("/home/grooveek/grails-1.1.1")
   fileToZip.zip("testfiles/grails.zip")
   def zipToUnzip = new File('testfiles/grails.zip')
   zipToUnzip.unzip('testfiles')
 }
}

I hope you enjoyed reading
Don't forget to let a kind word (or something not so kind if that's your mind)you want

See you soon

@grooveek

What strikes me in Groovy ...

2009-09-06T23:52:00.001+02:00

// Simple list with names.
def names = ['groovy', 'grails', 'mrhaki']

// Simple closure.
names.each { println 'Normal closure says: Hello ' + it + '!' }

// Groovy method to convert to closure.
def groovySays(s) {
 "Groovy says: Hello ${s}!"
}
// Use .& syntax to convert method to closure.
names.each(this.&groovySays)

// Convert Java method to closure and use it.
def javaSays = JavaObject.&javaSays
names.each javaSays

This code comes from mr haki's blog. What I like here is the ability to switch from Java to Groovy ! You can have the power of Java with the simplicity of Groovy
(and that's also a test post for the syntax highlighting plugin)

Welcome to this blog

2009-09-06T15:28:00.000+02:00

I love Groovy and Java. I mean : I LOVE Groovy and Java. I'm writing here to share experiences, and to get some help sometimes as I'm not a Groovy Champion ;-)
Feel free to share your feelings about Groovy and Java technologies here and please, please.... excuse me for my bad english. I'm French and would be really happy to find people who want to talk about what's really fun in coding !

My best JVM-related technologies at this time (just) :
* Groovy / Grails /Gorm
* Spring
* Gridgain
* HtmlUnit / Webtest

Enjoy your readings

Grooveek