Wednesday, July 29, 2009

CF: ColdFusion Serialization via Java API

This is a topic that has found many posts. Here is my spin ;o) Most of the posts focus on the success of serializing ColdFusion components. Though cool, many ask the same question: Why do this?
Also to note is that ColdFusion has supported a mechanism to serialize complex data via WDDX for a long time (I believe since version 4).
However, in my case, the need was not for serialization components but rather for compact serialization. In other words, use as little space (bytes/data) as possible. In addition, I needed to easily save and retrieve this information from a database in a text based format. Oh, yes, and handle complex data objects such as structures and arrays.
WDDX, though usable, is very verbose and thus was out. Looking at the Java API and reading through posts I translated this, in the end, to two functions fSerialize and fDeserialize. You will need ColdFusion 8.0.1 or higher to make this work.

The fSerialize function:

<cffunction name="fSerialize" access="public" returntype="string"
hint="uses java byte streams and Base64 encoding to serialize CF objects, this can be used instead of WDDX tag CFML2WDDX">

<cfargument name="input" type="any" required="Yes" hint="the CF object to be serialized">
<cfscript>

var objByteStream = createObject(
"java", "java.io.ByteArrayOutputStream").init();
var objOutStream = createObject(
"java", "java.io.ObjectOutputStream").init(objByteStream);
var objSerialized =
"";
//turn CF object in argument to out stream
objOutStream.writeObject(Arguments.input);
objOutStream.close();
//take outstream and make bytearray
objSerialized = objByteStream.toByteArray();
//encoded it and return

return BinaryEncode(objSerialized,"Base64");
</cfscript>
</cffunction>


fDeserialize function:


<cffunction name="fDeserialize" access="public" returntype="any"
hint="uses Base64 encoded Java Byte Array and turns to CF object. This can be used instead of WDDX WDDX2CFML">

<cfargument name="Input" type="string" required="Yes" hint="the Base64 encoded ByteArray that used to be a CF object to be deserialized">
<cfscript>

var objSerialized =BinaryDecode(Arguments.input,
"Base64");
var inByteStream = createObject(
"java", "java.io.ByteArrayInputStream").init(objSerialized);
var objInStream = createObject(
"java", "java.io.ObjectInputStream").init(inByteStream);
var objCF = objInStream.readObject();
//return the read object
return objCF;
</cfscript>
</cffunction>

These function on average were using 50% of the storage that a comparable serialized WDDX object would, so they achieved their objective for me. They will work on small components (cfc) but I have not tried to serialize very complex cfcs.

Cheers,
-Bilal

Wednesday, May 20, 2009

VSTS 2008: Database Load and Unit Test using Load Agents

There had to be a secret to this. Reading through the documentation, looking at examples, load testing with VSTS 2008 seemed a sophisticated affair, alas, only if you are working along the most popular path. More specifically only for web application testing.
I, being a renegade, wanted to use it to do Database Load testing. There are great new facilities in VSTS 2008 to create Database Unit tests but none speaks of how to do a load test.
I thought it would be easy enough to adapt the techniques in web-testing to database load testing. I found out I was very naive. I should have caught on when:
a) There was no documentation at all, no video, no web post, no nothing from MS or anyone having done this.
b) The number of errors I was getting when even trying to run it with one agent.

My test setup initially:
a) One workstation running controller and VSTS 2008
b) One server running VSTS Load Agent
c) A beefy SQL Server 2008 box to be load tested

Load testing on my workstation was working great, everthing was as expected and I was hopefull that this could easily translate into working well on the agent system. In short, it did not:

Here is the initial error I got:
  • System.IO.FileNotFoundException: Could not load file or assembly
I thought the deployment was enabled automatically, playing with the deployment option on the Test Run Configuration was not very helpfull. I ended up copying the assembly files into the same directories on the Load Agent, only to get errors referencing core related assemblies of VSTS2008. So, I created install project to deploy those... you guessed it, a wild goose chase in which I tried so many different things only to end up banging my head against a concrete wall. I nearly concluded that VSTS 2008 could not be used to do load testing for a database with Load Agents when an idea struck. This one ended up working, but could have been easily documented. So Microsoft peeps take it to heart ...

I changed the test rig in this fashion:
a) Workstation with VSTS 2008 and controller + SP1
b) Server with VSTS 2008 and Load Agent + SP1
c) SQL Server 2008

The solution, thus, is to install visual studio team system 2008 on the agent computer/server as well, then install the agent. What good is an agent that does not cover all test cases? That is something that Microsoft hopefully will resolve. There seem to be dependencies on VSTS core files, I played with it a little. You do not need:

a) Any Crystal stuff
b) SQL Server Express
c) C++
d) dotfuscator stuff
e) Documentation or redistributables

My test where in C# so I installed on support for it and no other language.

Cheers,
Bilal

Wednesday, April 29, 2009

Java: Connecting to MS SQL server with JDBC and JRE 1.6

So,
I have this little Java project which requires that I connect to SQL Server 2005. Something I have done many times. Download the JDBC drivers from MS and copy them into the lib directory and do your shindig in the connection to get going.
But, this time things have changed and the installation instructions are worthless at best.

Here is what I did:
  1. Downloaded the driver files from MS for JDBC 2.0
  2. Unzipped to directory
  3. Moved jar files (there are two files sqljdbc.jar and sqljdbc4.jar) to my lib directory
  4. coded my connector to use Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver")
  5. Did some query code

And presto, nothing works as expected. Instead I get an exception with this in the stack trace:

"The JDBC Driver version 2.0 does not support JRE 1.4. You must upgrade JRE 1.4 to JRE 5.0 or later when using the JDBC Driver version 2.0. In some cases, you might need to recompile your application because it might not be compatible with JDK 5.0 or later. For more information, see the documentation on Sun Microsystems Web site"

Complete bogusness as neither Microsoft or Sun contain much info about how to solve this. So, head scratching starts followed by experimentation.

Come to find out I made a mistake by copying both jar files packaged in the driver to lib. They contain similar classes and the sqljdbc.jar files classes' are used if both are copied. However, the sqljdbc.jar file classes are not ment to be used with jre/jvm 1.6+ . Why noone mention this in the provided install doc is a different question.

Solution:

a) Remove the not needed jar file (sqljdbc.jar) from lib directory or

b) specifically declare to use only the right one (sqljdbc4.jar)

Microsoft, please help us measly developers wasting time and state this in basic terms in your install doc.

There you have it.

Cheers,

Saturday, March 28, 2009

CF: Unable to perform cfflush in cffunction

OK. This is not an obvious one, so I decided to document this behavior.
Say you have several functions in components processing away happily, and you want to give the user some feedback to keep him/her entertained as well as keep their paws of the back buttons etc..
You think doing couple of flushes during your processing may be a good way until you try and get this error:

Unable to perform cfflush.

You have called cfflush in an invalid location, such as inside a cfquery or cfthread or between a CFML custom tag start and end tag.


What to do? Since none of the hints apply to you. Your code looks like this:

<cffunction name="fFlushIt" hint="flushes current content">
<cfargument name="feedback" default="">

<cfoutput>#Arguments.feedback#</cfoutput>
<cfflush>
</cffunction>

<cffunction name="fProcessing" returntype="numeric"
hint="does processing" output="No">

<cfset var x=10>
<cfset var y=20>
<cfset fFlushIt("Processing Complete")>

<cfreturn x + y>
</cffunction>

<BR>
<cfoutput>The result is: #fProcessing()#</cfoutput>


here comes the digging into the code and making guesses part. It seems that there is one more scenario cfflush won't like and this is if you call in within a call-tree (yes even nested stuff) in a function that has the output attribute declared as "No". Maybe a hint like that could be placed in the error to avoid all that brain scratching and wondering that goes along with this?
Anyways change the function like this and it worked. Yeah !



<cffunction name="fProcessing" returntype="numeric"
hint="does processing" output="Yes">




Cheers,

Thursday, March 26, 2009

Oracle: PLS-00103: Encountered the symbol ""

Triggers, triggers and more triggers.
Thousands of them. Ran the scripts into Oracle using ColdFusion instead of SQLPlus only to find out that I now had thousands of invalid triggers.
A little puzzling, as the same scripts worked like a charm everywhere else. I go use the oracle web enterprise manager to see whehter I can recompile them and make a few valid. Nada!
Everytime the Enterrpise manager tries to compile the trigger I get the error:

PLS-00103: Encountered the symbol ""

Here is a sample trigger with issues:

CREATE OR REPLACE TRIGGER INS_MySuperTrigger BEFORE INSERT ON
MySuperTriggerTable REFERENCING OLD AS OLD NEW AS NEW FOR EACH ROW
DECLARE
newid NUMBER;
BEGIN
If :new.MySuperTrigger_ID IS NULL THEN
SELECT SEQMySuperTrigger.NextVal INTO newid FROM dual;
:new.MySuperTrigger_ID := newid;
End If;
END;


This is nuts. I use the generate SQL button of Enterprise manager to generate the SQL and copy and paste it into the Oracel SQL Developer UI, run the code without mods, and bingo; the trigger is valid and happy as a peach.

What gives? Long hours wasted with different websites and options, casing, single and double quotes, Egyptian prayer beads..., you name it, I tried it.
Until pulling a protocol sniffer to see what the difference is between Oracle Enterprise Manager and Oracle SQL Developer on the wire.

Come to find out Oracle does not recognize Windows CRLF (Chr(13) + Chr(10)) as blank space, if you replace all the CRLF with LF this works like a charm. Seemingly, web based Oracle Enterprise Manager does not do this translation, while the Oracle SQL Developer tool does. Yack! Lesson learned, I now run the scripts through a parser before running them to Oracle via JDBC and get valid triggers all the time.

Cheers,

Tuesday, March 24, 2009

MySQL: frustration with login at the command prompt or the space character mystery

Many years ago when trying to use MySQL I ran into this, but only get to note it down now because someone else run into this problem. I searched and lo and behold not much info out there, so I thought maybe a couple of pointers would be worth while.
When using the command line mysql tool you can specify the username and password of the user during login.
Many manuals describe the syntax as:

mysql -h hostname -u root -p password
or
mysql --user=user_name --password=your_password db_name

You think wow this is easy, right. I can interpret this very simply.
I go to the command prompt and type:

C:\>mysql --user=root --pasword=mypass

but to my astonishment I would get:

mysql: unknown variable 'pasword=mypass'

Here are couple of other beauties that are in the manuals and don't work at all like expected:

C:\>mysql -u root -p mypass
Enter password: ******
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)

This could be very frustrating to someone new to MySQL on windows. Can't even login to get the client program started.

The approach I found working for windows based installation is to ignore the manual and avoid space between parameters.
Thus rather than using -u root, you will need to use -uroot. This will magically work. E.g.:
C:\>mysql -uroot -pmypass
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 19Server version: 5.0.67-community-nt
MySQL Community Edition (GPL)
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>


Hope this will avoid some frustration out there.
Cheers,

Thursday, March 19, 2009

CF: ColdFusion functions and the case of the mysterious space

Spent lots of time trying to find out why a space was returned from a function call to a function in a component. We would place the output directly into a html form text control, but a space would always be added to the front of the expected return string.
Broke out the old Hex editor to save and check file contents. Output each character at a time, just could not see why this would happen.
Many hours later, a headache started to kick in. Time for a break. Then, a hunch started knocking at the back of my scull and got louder and louder.
To prove it to myself, I created an intermediate variable into which I stored the function result. When this scheme was used, no space. If I used the function output in place, space! It turned out to be the Output attribute of the function definition. You will need to set it to 'No'. If you leave it off and attempt to use the function in place, Coldfusion will introduce a space in the return. Very, very annoying.

Here is sample code to reproduce the problem, two function that are identical except one has the output attribute specified; if you use the function 2 inline CF will produce a space in front of the output; see the output for f2 below:


<cffunction name="fFunction1" output="No">

<cfset var strReturn ="Hello World">
<cfreturn strReturn>
</cffunction>

<cffunction name="fFunction2" >
<cfset var strReturn ="Hello World">
<cfreturn strReturn>
</cffunction>

<cfset sOutput1=fFunction1()>
<cfset sOutput2=fFunction2()>

<cfoutput>
f1:--#fFunction1()#--
<BR>
o1:--#sOutput1#--
<BR>
f2:--#fFunction2()#--
<BR>
o2:--#sOutput2#--
<BR>
</cfoutput>

Output:

f1:--Hello World--
o1:--Hello World--
f2:-- Hello World--
o2:--Hello World--