How to authenticate to grails from R using RCurl

The trick is to open the login page before actually trying to post the login data. I assume something is stored in the cookie, but I only know this is how it works. See also my question on stackoverflow: http://stackoverflow.com/questions/20355874/how-to-use-rcurl-to-authenticate-to-spring-security-grails-app/20370222#20370222

My simple approach to security tokens

I'm a big fan of the spring-security plug-in series for grails. It allows me to do almost anything concerning security issues while giving me faith that this works properly. One scenario I encountered now, however, was not covered. In order to secure mit JSON web-service actions I couldn't rely on spring security. Instead I wanted to create unique links just like in google's picasa, when you wanted to share an image with a 3rd party that does not necessarily have a picasa account like you do. I decided to implement this in a very simple fashion: I added a field String uuid to the domain class in question. Then I added a service class: Then all I needed to do is to modify controller actions like this: I would like to hear what you think about this approach. Is it save enough? Is there a more elegant solution?

Pitfall with grails projections

With grails projections one should keep in mind that when one of the associated values turns out to be null, the entire entry is ignored in the result list. I found numerous posts, where this problem is addressed by alternating the join mode from inner join to left outer join:

http://www.intelligrape.com/blog/2011/11/01/criteria-query-with-left-outer-join/
http://stackoverflow.com/questions/17083204/criteria-uses-inner-join-instead-left-join-approach-by-default-making-my-que
http://stackoverflow.com/questions/19390720/grails-2-x-createcriteria-or-doesnt-work-for-nested-associations/19391255#19391255

Now, I do something like this:

Speeding up JSON export in Grails

In one of my grails apps, I export a large quantity of data in JSON format. This process was incredibly slow (~30s) and somehow I was convinced that this was a database issue since numerous joins were involved as well. As it turns out, however, the database query took ~1s to complete so that I got curious what the other 29s were spent on. Very unexpectedly, I had to blame grails' native JSON converter. A google search revealed that this was already known:

http://sysgears.com/articles/speedup-json-parsing-grails/

I decided to follow the suggested solution and started playing around with Jackson. Pitfalls are that some Java objects seem to cause Jackson some trouble, e.g. if objects can't be serialized. I therefore decided to use projections to efficiently recover all the database fields I was interested in (probably speeding up the database access as well). Since all the fields were then strings, ints or doubles, conversion worked like a charm. The exact same conversion now takes only ~1s, adding up to ~2s in total.

This helped me to understand how to use Jackson:
http://wiki.fasterxml.com/JacksonInFiveMinutes

There even seems to be a grails plug-in available to replace the default JSON converter with Jackson:
https://github.com/sjhorn/grails-jackson

However, I didn't try it out since development does not appear to be active.

One last important thing to note is that if you want to use jackson to render JSON content you might get into conflict with the regular JSON converter if you try to do something along the lines of render AS JSON. What you need to do instead is:

Deploying a gails app with the cloud-foundry plug-in

I recently discovered that CloudFoundry accounts are free (at least for now) and this offers a perfect opportunity to install a demo instance of grails applications. As with most things in grails, this is pretty straight-forward if you forget about the one or two issues you need to solve beforehand. I basically followed Burt Beckwith's excellent guide here: http://blog.springsource.org/2011/04/12/one-step-deployment-with-grails-and-cloud-foundry/ There were only two things giving me a headache. One issue was that the application did not get enough memory (PermGen of course). This was solved easily by reading the documentation of the plug-in:

//increase memory size to 1G (2G is max), don't start immediately since we need to change JAVA_OPTS
grails cf-push --memory=1G --no-start
grails cf-env-add JAVA_OPTS "-Xmx1024m -XX:MaxPermSize=512m -XX:+CMSClassUnloadingEnabled"
grails cf-start

The second issue became apparent from the exception: liquibase.exception.ServiceNotFoundException: Could not find implementation of liquibase.logging.Logger Thinking about this for a second I remembered that Liquibase was used by the database migration plug-in. I just removed it from the runtime in BuildConfig.groovy and tadah everything was running fine! I also raised a jira for this: http://jira.grails.org/browse/GPCLOUDFOUNDRY-52

jQuery together with other JS libraries (e.g. Prototype)

I want to gradually migrate from Prototype to JQuery in one of my projects (grails 2.0.4), but when I tried to use both libraries at the same time I got some error messages and the JS functions did not work anymore. Turns out JQuery has a compatibility feature you can turn on: http://docs.jquery.com/Using_jQuery_with_Other_Libraries jQuery.noConflict() On top of that there were issues with the resources plugin, which I circumvented for now by manually loading jQuery where I need it.

grails database migration plugin: upgrading an existing database

Today was in fact the first time I ever got into a situation where the setting dbCreate = 'update' couldn't manage to update my data base scheme after modifying some of my domain classes. This was unfortunate of course, but since I've been reading that this is bad practice anyway I decided to have a closer look at the new database migration plugin that came out at roughly the same time as grails 2.0 did. This plugin utilizes Liquibase, which has been around for a while from what I could see and seemingly does a very good job. It turns out that the database migration plugin gives you a really smooth integration of Liquibase and grails.

Coming to my actual problem:
After adding the spring-security plugin I wanted to have some light history with two of my domain classes, so I introduced the fields

• createdBy,
• lastUpdatedBy,
• created,
• lastUpdated.

The latter two are automagically populated by GORM and the first two I could populate in the controller classes using the springSecurityService. This obviously leaves existing database rows without this information and since I assigned a "not nullable" constraint, hibernate sure enough complaint when trying to change the scheme.

So I did some reading on the excellent tutorials I found to get up to speed:
http://blog.springsource.org/2011/08/17/countdown-to-grails-2-0-database-migrations/
http://fbflex.wordpress.com/2011/01/19/working-with-the-grails-database-migration-plugin/

After understanding the key concepts I went ahead and followed the instructions at the end of the second blog post (updating an existing database). I created a new environment "migrate" which I configured to use a copy of my database. My domain classes where already changed, so I just needed to compare the current database scheme with my domain classes using

This yields a changelog.groovy file that translates the liquibase xml configuration into the groovy world. Since the conversion is straight-forward you can make use of the liquibase docs to modify this changelog. The original changelog corresponds exactly to what GORM would have done anyway with dbCreate="update".

You can try and run it using the dbm-update command:
You might be happy at first seeing that everything seems to work fine, but soon you'll be frustrated because nothing actually happened. At this point you might have already asked yourself (like me) how the plugin knows which changelog you want to apply. And in fact you have to specify this in Config.groovy: Of course running the update failed for me. I didn't do anything about the not null constraint. I solved this problem by removing the constraint from the original add column configSet and adding a new configSet add the end where I specified a defaultNullValue ( http://www.liquibase.org/manual/add_not-null_constraint ), which replaces all my trouble-causing NULL entries with a sensible value: This worked really fine for the createdBy and lastUpdatedBy column where I only had to set an integer value. However, it took me quite some time to figure out how to set a date correctly. I read a lot about timestamp and getdate() functions and in the end the simplest solution was to just figure out the date syntax which looks like this:

 A few important tips for working with this plugin:

• Use grails -Dgrails.env=yourEnv to run a custom environment besides dev, test or prod.
• Use grails dbm-update-sql to see what SQL code is created.
• Be aware: The dbm-update command often fails more or less silently. Therefore: 
• Look into your database! There is a new table called DATABASECHANGELOG. 
If you compare the entries in this table you will quickly see which configSet caused the operation to fail. This is most likely due to a syntax error because problems with the database usually cause an exception.

And if you finally get to see the following line:


| Finished dbm-update

You finally did it and you can leave for the week-end. 

Select from a map in GSP?

Logging in grails 1.3.7

Logging capabilities haven improved a lot in grails 2.0, but my main project is still running on grails 1.3.7 and somehow I never managed to get any of the profiling plugins to work. I was suspecting that my application made a lot of repetitive and unnecessary requests to the SQL database so I needed a way to figure out what was happening between hibernate and the JDBC connection. Hibernate comes with its own logging but unfortunately it doesn't tell you anything about the response time of the database. So I was digging google and found this very promising approach on http://piraguaconsulting.blogspot.com/2010/06/grails-p6spy-and-sql-profiler.html, where p6spy basically replaces your database driver with a shadow version that logs all communication. Unfortunately this didn't work for my project. I use a C3P0 connection pool and the shadow driver of p6spy can't handle that (at least not as far as I could see). A pity since it seemed to be a good idea especially in combination with http://sourceforge.net/projects/sqlprofiler/ which combines the logging with a nice GUI.

However, this approach gave me a basic idea of what I was looking for and googling deeper I found http://code.google.com/p/log4jdbc/ which in contrast to p6spy is actively developed. Hooking it up with grails is super easy. You add the lib file to your project, exchange the database driver in your dataSource config with

net.sf.log4jdbc.DriverSpy

and adjust your database url:

Before:
jdbc:sqlserver://localhost:1433;databaseName=olabdb;

After:
jdbc:log4jdbc:sqlserver://localhost:1433;databaseName=olabdb;

Simple as that. The shadow driver of log4jdbc seems to recognize the most common SQL drivers and includes them without any effort from my side.

In order to turn logging on you add 'jdbc.sqltiming' to you log4j config in Config.groovy, e.g.
Log4j config is also explained here: http://grails.org/doc/latest/guide/conf.html#logging

Now it already works and gives you a nice and informative output including the parameters that are actually part of the SQL query. This helps a lot to learn how your domain model is reflected in the SQL world.


2012-03-08 10:58:10,515 [http-8080-5] INFO  jdbc.sqltiming  - select genes0_.recombinant_genes_id as recombin1_0_, genes0_.gene_id as gene2_0_ from gtRecombinant_gtGene
genes0_ where genes0_.recombinant_genes_id=977


My lesson was that inheritance is actually a bad thing since it causes multiple joins for each query that you do on a child class. Another thing I don't like is how hard it is to read the statements since all column names are referred to by a different name. A little more digging and I found this really nice post:

http://softdevbuilttolast.wordpress.com/2010/02/22/hibernate-sql-logging-log4jdbc-haciendo-log-de-sql-con-hibernate-log4jdbc/

It teaches you how to configure log4jdbc efficiently and to tailor it towards your needs. In my case I added a few starting parameters to my VM:

-Dlog4jdbc.debug.stack.prefix=org.openlab

This one allows you to specify your applications main package, which in turn allows log4jdbc to see where your own code begins. As a result it tells you which line of code caused the SQL query. SUPER NICE. Note that it only seems to work on log level 'all'.

Among hundreds of queries you want to see the "bad guys" that just take too long. You can set time thresholds for log.warn and log.error via

-Dlog4jdbc.sqltiming.warn.threshold=200
-Dlog4jdbc.sqltiming.error.threshold=1000

IDEA 11 and grails 1.3.7 / 2.0

Last week I gave IntelliJIDEA a shot at grails since we now have both, a new version of grails (2.0) and IDEA (11) which has a reputation for good grails support in the ultimate edition. Creating a new grails project with grails 2.0 was no problem at all. Although you have to inconveniently download and extract grails yourself, everything else works like a charm. IDEA has very good in-build support for grails, meaning you can install plugins via IDE etc. It provides a dedicated grails view for your projects and thus is offering kind of the same support as Springsource's own IDE called STS. A feature for building different kinds of grails classes is also given. What I particularly like is that you can create controller, views and tests directly via buttons offered on top of the domain class editor view. Just great :-)

But so far we haven't seen anything worth what Jetbrains charge for their software. This only becomes apparent if you use it for a little while. In the process you'll learn that code completion is by far superior to STS. IDEA always knows exactly what your domain classes look like and offers the right completion for dynamic finders à la findAllBy*. Cool stuff, but most convincing to me is actually the very good reaction time of IDEA compared to STS with mostly sluggish behaviour and an auto-update function that never worked for me. Bottom line for me was that I know decided to switch to IDEA.

For people moving from STS to IDEA, another thing will be most important. How difficult is it to move existing grails projects to IDEA? I mainly work on one project upgraded to grails 1.3.7 that involves a number of inplace-plugins. I told IDEA to create a new project from existing sources, selected the encompassing folder of all my projects and clicked next. IDEA recognized all my projects as grails/groovy projects, so I just had to select grails. Next, it asked for the SDK to use. Instead of using the 2.0 grails I added a new SDK pointing to the grails 1.3.7 folder of STS. After that, IDEA recognized all my plugins correctly (my former projects are now modules) and my application works out of the box with a normal startup (mainly thanks to using the same grails installation that already has downloaded all the necessary plugins). I was astonished how well that actually went :-)

obtain class with only short class name at hand

If you use packages in grails it's sometimes not easy to resolve a domain class, e.g. if you only have the short class name "gene" in your params when you would actually need "package.name.gene". Here is how I managed to resolve the problem:

grailsApplication.getArtefactByLogicalPropertyName("Domain", domainName.toString().toLowerCase())

how to ask user for confirmation before executing remoteLink

before:"if(!confirm('Are you sure?')) return false"

The trick is not to return "true". Credits go to this jira.

short reminder for stupid me

If the tag does not work, it is most probably because I forgot to add the braces to the variable:

never works!

usually does work!

grails and SVN

I added my grails projects to a SVN repository, in order to be able to work someplace else. When I tried to check my projects out however, they did not run.

I included .project and .classpath files as I already learned that they are needed for eclipse to recognize them as grails projects.

Problem: When I ran run-app I got error messages complaining that plugins could not be resolved (locally). Also the plugin manager in STS did not list any plugins. Nothing worked.

What I missed however was that you have to run grails upgrade to reinstantiate other missing files. Afterwards plugins are resolved correctly and downloaded from the internet. This solution was found here.

preventing grails-ui tooltip from disappearing

Today I tried to prevent the grails-ui tooltip from disappearing after 5 seconds when the mouse is not moved. From the YUI doc I knew I had to alter a parameter 'autodismissdelay', but I haven't found a way to pass this parameter with the taglib. Unfortunately the only way I succeeded was to alter the ToolTip.js in grails-ui-1.2-SNAPSHOT\web-app\js\grailsui where I changed the value from 5000 to -1.

making export plugin work with filterpane

The export plugin and the filterpane plugin for themselves work like a charm. If you try to export your filtered list however, you run into problems. The root of this problem has been recognized during an earlier discussion on the mailing list.

The problem: You have to extend the filter action to deal with the export. Therefore you have to apply the filterService.filter method with the same filter params as when you applied the actual filter.

The workaround (not a perfect solution): To solve this problem I save the filter params in the session. When export is applied on the filter action the filter params are retrieved from the session. That works well although I'm not very happy with storing so much information in the session permanently.

The Code:

Here I have modified the example from the plugin page. The export-part of the list action is extracted and gets its own export closure, which can then be used by the filter action as well. I also do not explicitly name the parameters as I am using this code in a scaffolded controller. Therefore a list is fetched of all properties, excluded those that are not really representative.

def export = {attrs ->
  
  def response = attrs.response
  println attrs.exportList
  
  def excluded = grails.persistence.Event.allEvents.toList() + ["mapping", "lastModifierId", "hasMany", "class", "belongsTo", "constraints", "searchable", "attached", "errors", "metaClass", "log", "object", "version", "beforeInsert", "beforeUpdate", "mappedBy", "springSecurityService", "type", "typeLabel"]
  List fields = ${className}.newInstance().properties.keySet().toList().findAll { !excluded.contains(it) && !isId(it)}

  response.contentType = org.codehaus.groovy.grails.commons.ConfigurationHolder.config.grails.mime.types[attrs.format]
  response.setHeader("Content-disposition", "attachment; filename=${className}.\${attrs.extension}")
  
  exportService.export(attrs.format, response.outputStream, attrs.exportList, fields, [:], [:], [:])
 }
 
 def isId(def label)
 {
  label.reverse().startsWith("dI")
 }

 def filter = {  
  if(!params.max) params.max = 10
  println "out:" +params
  if(params?.format && params.format != "html" && session.filterParams)
  {
   def exportList = filterService.filter( session.filterParams, ${className} )
   println "list:" + exportList
   export(response: response, extension: params.extension, format: params.format, exportList: exportList)
  }
  
  session.filterParams = params
  
  render( view:'list',
   model:[ ${propertyName}List: filterService.filter( params, ${className} ),
   ${propertyName}Total: filterService.count( params, ${className} ),
   filterParams: com.zeddware.grails.plugins.filterpane.FilterUtils.extractFilterParams(params),
   params:params ] )
 }

EDIT: Steve made me aware that it is easy to extract the necessary parameters with a helper method. One should therefore rather modify the export taglib to pass the correct parameters to the filter action. Also see Steve's comment below.

EDIT: Please forget about most of the crazy stuff I have done here :-) You can pass the filterParams with the export tag like this:

just some notes about working with scaffolding code

When I modify scaffolding templates I always wonder how to obtain different properties like e.g. the class name or the id of the current instance. A general remark: What confused me most in the beginning was notation like \${${...}}. What you have to know is that \$ is escaped and is therefore what you can normally achieve in GSPs with a $. The unescaped $ within that clause is for the template. When actual code is generated from the template those $ are resolved immediately. You can therefore only use properties that are kown at this stage. Here are a few examples.

\${${className}}

This resolves the long version of the current class name

You often see that properties of an instance are iterated. The variable name of a property is then 'p':

${p.name}

This is the name of the property as it is used in the current instance. You need this value if you want to access that property in a controller (similar to this: domainClassInstance."${params.propertyName}").

${p.referencedDomainClass?.propertyName}

This one gives you the propertyName of the referenced domain class, thus the domain class of the property p. The propertyName is equal to the controller name if you stick to the convention.

${p.referencedDomainClass?.fullName}

Sometimes you need to have the full class name with the whole package notation. This can be useful for domain class instantiation (see my last post).

${p.referencedDomainClass?.shortName}

This is the actual class name omitting the package notation.

how to get a domain class instance from a class string (e.g. via params)

I had the problem that I could pass the class name from my gsp to my controller, but I didn't know how to work with that being a String. Luckily I stumpled about the solution in a code example somewhere on the web:

def grailsApplication
grailsApplication.getDomainClass(params.className).newInstance()

creating custom grails artefacts

I wanted to create my own kind of artefacts, in order to make easy extension of my application easier. I already had an interface called "Module" that has been implemented by classes that add new content to my sites. There were two problems that artefacts could solve for me:

1. Dynamic reloading on code change won't work in non-artefact classes :-(
2. Whenever I introduce a new module, I have to manually register it with a service handler.

As an artefact a module is automagically added and corresponding classes are hot-swapped if changed. I followed the great instructions on this page to get started.

The only thing I have done differently was in the last step, when I access the modules like this:

def grailsApplication

def modules = grailsApplication.getModuleClasses().collect { 
 it.referenceInstance
}

The thing is, getModuleClasses() returns 'GrailsClasses' and not java.lang.classes. At first I took the wrong road from there, trying to use it.class, which is then the DefaultModuleClass that you had to define with no abilities. I wondered how to obtain the underlying 'real' module class. Looking into the properties of the GrailsClass I found out that you can get the class of the module by accessing 'class'. But you can do even better: A instance is already attached and can be accessed by it.referenceInstance.

Happy artefacting!

closures in bootstrap.groovy not working

Just a note: Closures are currently (I believe since 1.2.2) not working in Bootstrap.groovy. This is also documented in JIRA. What you can do however, is use methods instead.