Pretty Good Privacy (PGP) is a computer program that provides cryptographic privacy and authentication.
PGP is often used for signing, encrypting and decrypting e-mails to increase the security of e-mail communications. It was created by Philip Zimmermann in 1991.
The use of PGP in file exchanges has increased over the years and has become a common way of securing file contents. Thus, encountering this in more and varied projects is not unusual.
There are several implementations for popular platforms such as Java and .net; I classified them into brought camps of heavy commercial, or hard to use open source; thus the availability of easy to
implement PGP solutions for ColdFusions is limited. In particular, I found existing implementation rather difficult to use.
Thus, I embarked on this project. To make a long story short, I ended up using the underlying work of the
league of bouncy castle folks. The complete source code for those libraries can be downloaded from there.
I implemented the most common scenarios (generate keys or key rings, encrypt and decrypt) and exposed all this through a ColdFusion component to the world. Thus the effort to use of PGP is reduced to one liners of code (which I like).
Along the same lines, if you use Flex and needed a library to encrypt/decrypt this implementation can easily used as backend for that kind of scenario. A few tweaks to the main component should allow you to call on all the functions, read, write, generate keys etc.
Of course, there is so much more that can be done with PGP. Time permitting I may expand this implementation to expose other functionality.
You can download this from Github (https://github.com/Bilal-S/cfpgp/releases/tag/2.0.0).
Cheers,
-Bilal
Thursday, January 28, 2010
Thursday, January 7, 2010
CF: Connecting ColdFusion to Microsoft Azure SQL
I am impressed by what Microsoft has been able to create with the Azure platform. The more I play with it the more I like it. What impresses me most is how simple it is to get going (I am comparing with Amazon). The big caveat here is that you use MS technologies. If so, Azure should be on the short list for evaluation.
However, and, here I go again reversing myself, if you want to connect a ColdFusion front end to a Microsoft Azure SQL back end, things are a little more quirky. Please don't ask why you would do such a thing, just stay with me in that it is highly cool ;o)
First, you should evaluate whether your application will give to this kind of setup. On the ColdFusion side you should have good control on the amount of data you exchange with the database(pagination and other techniques); you also should have some tolerance for higher latency; there is delay in the round trip to get data from Microsoft etc.
Of course the usual evaluation of whether you are ok with hosting data in the cloud should take place. But on the other hand, taking this for a spin is easy enough.
In my setup I am hosting the CF server on my laptop and MS is supplying the database.
1) Learn about Azure: Learn about Azure things and then Create Account
2) Create your first database instance online using your browser
3) Get your tools. You do not need to download the Visual Studio or other SDKs to work with SQL Azure, but you will need to use the SQL Server 2008 R2 Management tools. Currently only available as CTP edition.
4) Optional: Migrate some data. For me, using the script wizard in the Management Studio worked the best. You can switch it to produce SQL Azure specific scripts, but you still have to tweak them. Your primary keys have to be clustered when creating tables.
e.g.:
CREATE TABLE [dbo].[TestTable](
[Test_ID] [int] NOT NULL,
[TestName] [nvarchar](50) NOT NULL,
[TimeZoneOffsetHours] [smallint] NULL,
[TimeZoneOffsetMins] [smallint] NULL,
CONSTRAINT [PK_TestTable] PRIMARY KEY CLUSTERED (
[Test_ID] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF)
5) Once you have some data we start with the ColdFusion parts. I have not been able to use the Adobe supplied drivers. I had to download the Microsoft JDBC 2.0 drivers. When you extract the jdbc drivers there are two .jar files. For CF you will need to use [sqljdbc4.jar]. Copy this driver into [cfroot]\lib folder.
6) Restart ColdFusion
7) Configuring your Data Source in the ColdFusion administrator. Here it gets a little complicated. You need to create a data source of type "other". Also, you cannot use the connect strings that are supplied by MS directly, though pieces are useful. The current stance is that Microsoft does not support JDBC connection but if you can work it they are fine with it. To make it simple I am attaching an image: (please note the way in which we have to set the User name)(replace myID with the database User ID and myServer with your server name from Azure):
8) Configure your connection settings. The Azure platform will drop your connection frequently. So maintaining connection is not really an option. This is what worked best for me to make it appear as if we had no connection issues.
Ok, hopefully this all worked out for you and you can take this for a spin.
Cheers,
Bilal
However, and, here I go again reversing myself, if you want to connect a ColdFusion front end to a Microsoft Azure SQL back end, things are a little more quirky. Please don't ask why you would do such a thing, just stay with me in that it is highly cool ;o)
First, you should evaluate whether your application will give to this kind of setup. On the ColdFusion side you should have good control on the amount of data you exchange with the database(pagination and other techniques); you also should have some tolerance for higher latency; there is delay in the round trip to get data from Microsoft etc.
Of course the usual evaluation of whether you are ok with hosting data in the cloud should take place. But on the other hand, taking this for a spin is easy enough.
In my setup I am hosting the CF server on my laptop and MS is supplying the database.
1) Learn about Azure: Learn about Azure things and then Create Account
2) Create your first database instance online using your browser
3) Get your tools. You do not need to download the Visual Studio or other SDKs to work with SQL Azure, but you will need to use the SQL Server 2008 R2 Management tools. Currently only available as CTP edition.
4) Optional: Migrate some data. For me, using the script wizard in the Management Studio worked the best. You can switch it to produce SQL Azure specific scripts, but you still have to tweak them. Your primary keys have to be clustered when creating tables.
e.g.:
CREATE TABLE [dbo].[TestTable](
[Test_ID] [int] NOT NULL,
[TestName] [nvarchar](50) NOT NULL,
[TimeZoneOffsetHours] [smallint] NULL,
[TimeZoneOffsetMins] [smallint] NULL,
CONSTRAINT [PK_TestTable] PRIMARY KEY CLUSTERED (
[Test_ID] ASC
)WITH (STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF)
5) Once you have some data we start with the ColdFusion parts. I have not been able to use the Adobe supplied drivers. I had to download the Microsoft JDBC 2.0 drivers. When you extract the jdbc drivers there are two .jar files. For CF you will need to use [sqljdbc4.jar]. Copy this driver into [cfroot]\lib folder.
6) Restart ColdFusion
7) Configuring your Data Source in the ColdFusion administrator. Here it gets a little complicated. You need to create a data source of type "other". Also, you cannot use the connect strings that are supplied by MS directly, though pieces are useful. The current stance is that Microsoft does not support JDBC connection but if you can work it they are fine with it. To make it simple I am attaching an image: (please note the way in which we have to set the User name)(replace myID with the database User ID and myServer with your server name from Azure):
8) Configure your connection settings. The Azure platform will drop your connection frequently. So maintaining connection is not really an option. This is what worked best for me to make it appear as if we had no connection issues.
Ok, hopefully this all worked out for you and you can take this for a spin.
Cheers,
Bilal
Thursday, December 24, 2009
CF: Parsing Complex Fixed (Flat) Files
Despite many years of XML, the use of simple and complex flat files as a means of exchanging information between systems is still fairly common.
Parsing simple, tabular, flat files like the one below does not really pose a large challenge as either the OS (Text Drivers) or CF (CFHTTP) have tools to read them in without much fanfare.
Example 1: Tabular Fixed File
However, many times over, the examples are not as simple. The data on each row may be different; relationships may exist between lines of data etc.
Example 2: Complex Fixed File
For the complex files we normally write specialized parsers or transformations. Certainly a whole industry exists that deals with Extract Transform Load processes. Either choose requires considerable programming efforts.
After writing a parser for the umpteens fixed file that someone wanted to have loaded, I thought there needs to be a better way. The short of all this is that I created a more generic component to deal with this in native ColdFusion. The solution was to split up the code and the file definition.
Thus, in order to parse complex files, you provide an XML definition file which describes the flat file. Then call the component to do the heavy lifting.
Example 3: Flat File Definition XML
The outcome of such processing are standard ColdFusion objects that are easier to deal with and require less programmatic effort to implement.
I posted this project on RIAForge (FixedFileReader). It contains examples for different scenarios of varying complexity. I added skeletons for EDI X.12 and VCF4 formats.
May not work for all, but if it does, it is likely that it will save a lot of time.
Cheers,
Bilal
Parsing simple, tabular, flat files like the one below does not really pose a large challenge as either the OS (Text Drivers) or CF (CFHTTP) have tools to read them in without much fanfare.
Example 1: Tabular Fixed File
However, many times over, the examples are not as simple. The data on each row may be different; relationships may exist between lines of data etc.
Example 2: Complex Fixed File
For the complex files we normally write specialized parsers or transformations. Certainly a whole industry exists that deals with Extract Transform Load processes. Either choose requires considerable programming efforts.
After writing a parser for the umpteens fixed file that someone wanted to have loaded, I thought there needs to be a better way. The short of all this is that I created a more generic component to deal with this in native ColdFusion. The solution was to split up the code and the file definition.
Thus, in order to parse complex files, you provide an XML definition file which describes the flat file. Then call the component to do the heavy lifting.
Example 3: Flat File Definition XML
The outcome of such processing are standard ColdFusion objects that are easier to deal with and require less programmatic effort to implement.
I posted this project on RIAForge (FixedFileReader). It contains examples for different scenarios of varying complexity. I added skeletons for EDI X.12 and VCF4 formats.
May not work for all, but if it does, it is likely that it will save a lot of time.
Cheers,
Bilal
Wednesday, November 18, 2009
CF: Controlling Time Based Runtime Exceptions
Ultimately time controls everything. This is true in larger scheme of things but as well in writing and running ColdFusion code. Fortunately, many CF tags have built in time controls in forms of exceptions that can be thrown when a pre-specified time has been exceeded. Most tags implement this through a timeout attribute.
You, then, use a CFTRY/CFCATCH combination to handle timeout occurrences, e.g. a web page failed to respond, or FTP didn't connect, etc.
However, this system breaks down if your intent is control multiple tags in combination, e.g. a combination of CFHTTP and CFFTP may take up to 30 seconds though individually CFHTTP or CFFTP may consume a variable amount of time.
Similarly, you are running a multitude of small steps in a loop, e.g. loading 1000 records, or displaying 500 lines. Though each line will only take a fraction of a second to process the number of lines in aggregate will blow you out of the water.
Normally in these cases we would wait on an overall page time out if we had one, then produce a generic catch all message saying your page has timed out and good luck trying this again.
This may suffice generically speaking, but may prove to be frustrating for users and visitors of your apps and sites.
What if you could instead be more pro-active. What if you could predict how long your processing is going to take, even on shared servers with varying loads?
What if you could give users options to either proceed, adjust your number of iterations, or simply stop early because you will never finish the task at hand in the time given.
Ideally your code would be very compact:
Conceptually speaking this could have been done for a long time in CF, but practically I have seen few implementations.
Let's look at what it would take to make this prediction using a scenario in which we have a repetitive core process that we need to time. Thus, it would be all about the ability to measure time for each loop. Thus we would
a) need to start a timer, e.g. GetTickCount()
b) end the timer, e.g. another GetTickCount()
c) measure the difference between start and end time
d) use the remaining iterations to come up with how much more time we need
Simple, right? Indeed it is. But the downfall is that it doesn't work.
The system breaks down because of the high variability in practice between each iteration of the loop. Using the built in CF function GetTickCount() we could establish a start and end time. But, when we measure the execution time between the two tick counts the results in this realm change quite a bit.
So, will we have to implement something complex using Java JETM instead to get this working? You probably could, but it isn't necessary if we add a dose of statistics to the whole puzzle. Here is a modified approach:
a) establish statistical validity threshold in number of iterations (e.g. how many time do I need to have gone through the loop to have a valid sample)
b) still need to start a timer, e.g. GetTickCount()
c) end the timer, e.g. another GetTickCount()
d) measure the difference between start and end time
e) update your statistics
f) if validity threshold has been reached make prediction
This does seem to work a lot better but requires many more lines of code. Rather than put all of these lines of code into each template, I have abstracted the needed calls into a component.
I would initialize the component (objTC) with the number of iterations (number of times I would need to go through the loop) and how many times I will need to have completed the loop to get a valid time prediction. In my example this would be 1000000 iterations with 150 completions. Or, something like this:
Then within the loop I would start and end the main timer like so:
End Measuring:
Now that the component takes care of all the nasty statistics stuff I can keep my main code simple and have some functions to use to either provide feedback to user, or just check on the status of things.
E.g. I could provide feedback like so:
This is just a sample on how to handle it. I have put several more functions in the examlpe to play with. It may work well for some of your scenarios or not all. In any case, I do believe an improved feedback to user and more predictive code behavior based on execution time will reflect well on you as programmer. So feel free to play with samples and see whether this can get you started.
The full sample code can be downloaded here.
Cheers,
-B
You, then, use a CFTRY/CFCATCH combination to handle timeout occurrences, e.g. a web page failed to respond, or FTP didn't connect, etc.
However, this system breaks down if your intent is control multiple tags in combination, e.g. a combination of CFHTTP and CFFTP may take up to 30 seconds though individually CFHTTP or CFFTP may consume a variable amount of time.
Similarly, you are running a multitude of small steps in a loop, e.g. loading 1000 records, or displaying 500 lines. Though each line will only take a fraction of a second to process the number of lines in aggregate will blow you out of the water.
Normally in these cases we would wait on an overall page time out if we had one, then produce a generic catch all message saying your page has timed out and good luck trying this again.
This may suffice generically speaking, but may prove to be frustrating for users and visitors of your apps and sites.
What if you could instead be more pro-active. What if you could predict how long your processing is going to take, even on shared servers with varying loads?
What if you could give users options to either proceed, adjust your number of iterations, or simply stop early because you will never finish the task at hand in the time given.
Ideally your code would be very compact:
<!--- check prediction --->
<cfif objTC.WillTimeout()>
<!--- we will timeout lets do something about it --->
...
Conceptually speaking this could have been done for a long time in CF, but practically I have seen few implementations.
Let's look at what it would take to make this prediction using a scenario in which we have a repetitive core process that we need to time. Thus, it would be all about the ability to measure time for each loop. Thus we would
a) need to start a timer, e.g. GetTickCount()
b) end the timer, e.g. another GetTickCount()
c) measure the difference between start and end time
d) use the remaining iterations to come up with how much more time we need
Simple, right? Indeed it is. But the downfall is that it doesn't work.
The system breaks down because of the high variability in practice between each iteration of the loop. Using the built in CF function GetTickCount() we could establish a start and end time. But, when we measure the execution time between the two tick counts the results in this realm change quite a bit.
So, will we have to implement something complex using Java JETM instead to get this working? You probably could, but it isn't necessary if we add a dose of statistics to the whole puzzle. Here is a modified approach:
a) establish statistical validity threshold in number of iterations (e.g. how many time do I need to have gone through the loop to have a valid sample)
b) still need to start a timer, e.g. GetTickCount()
c) end the timer, e.g. another GetTickCount()
d) measure the difference between start and end time
e) update your statistics
f) if validity threshold has been reached make prediction
This does seem to work a lot better but requires many more lines of code. Rather than put all of these lines of code into each template, I have abstracted the needed calls into a component.
I would initialize the component (objTC) with the number of iterations (number of times I would need to go through the loop) and how many times I will need to have completed the loop to get a valid time prediction. In my example this would be 1000000 iterations with 150 completions. Or, something like this:
<cfset objTC = CreateObject("component","TimeControl").init(1000000,150)>
Then within the loop I would start and end the main timer like so:
<!--- start measuring --->
<cfset objTC.start()>
End Measuring:
<!--- end measuring --->
<cfset objTC.end()>
Now that the component takes care of all the nasty statistics stuff I can keep my main code simple and have some functions to use to either provide feedback to user, or just check on the status of things.
E.g. I could provide feedback like so:
<cfif objTC.HasValidAverage()>
<!--- send the expected time to user
when this process will complete --->
<cfoutput>
Expected completion at #objTC.getFinishTime()#
or (#objTC.getFinishSeconds()#seconds) <br/>
</cfoutput>
<cfflush>
This is just a sample on how to handle it. I have put several more functions in the examlpe to play with. It may work well for some of your scenarios or not all. In any case, I do believe an improved feedback to user and more predictive code behavior based on execution time will reflect well on you as programmer. So feel free to play with samples and see whether this can get you started.
The full sample code can be downloaded here.
Cheers,
-B
Saturday, November 14, 2009
CF: Meetup presentation about debugging materials (Nov 12, 2009)
Thanks again to everyone attending the presentation. The goal was to get an idea about debug space and think about attacking debug problems by first identifying what area they originate from. Hope this gave everyone to use.
Also we discussed with special focus on time based debugging from the performance perspective as well as from the perspective of handling time based exceptions predictevly.
Dowload presentation and code now.
Several questions regarding the use of CFDUMP came up and here are couple of links to deeper treatment of advanced CDFDUMP tag in CF9. Additinal attributes can be specified to tweak its behavior:
CFDUMP in CF9
Cheers
-Bilal
Also we discussed with special focus on time based debugging from the performance perspective as well as from the perspective of handling time based exceptions predictevly.
Dowload presentation and code now.
Several questions regarding the use of CFDUMP came up and here are couple of links to deeper treatment of advanced CDFDUMP tag in CF9. Additinal attributes can be specified to tweak its behavior:
CFDUMP in CF9
Cheers
-Bilal
Wednesday, October 21, 2009
CF: CFinNC presentations
Hallo everyone. First thanks for attending my sessions. If you did not attend and wanted to take a peak at what we covered please feel free to review the slides. Of course more fun was had during the sessions ;o)
Download Beginning Debugging Session
Download Advanced Debugging Session
I have inlcluded more slides and examples than we covered hoping that they can be usefull as well. All my material are distributed under creative commons license. Feel free to contact me with questions or comments.
Generally, this conference showed that the spirit of CF is alive and well. It was an all volunteer event from the organization to the speakers and all the things in between. It introduced old and new people to CF and furthered networking in this Region. I hope we can make this an annual event.
Cheers,
-B.
Download Beginning Debugging Session
Download Advanced Debugging Session
I have inlcluded more slides and examples than we covered hoping that they can be usefull as well. All my material are distributed under creative commons license. Feel free to contact me with questions or comments.
Generally, this conference showed that the spirit of CF is alive and well. It was an all volunteer event from the organization to the speakers and all the things in between. It introduced old and new people to CF and furthered networking in this Region. I hope we can make this an annual event.
Cheers,
-B.
Tuesday, October 13, 2009
CF: CFinNC is all about debugging for me
My presentations for CFinNC will be all about debugging. There is a lot to talk about crunched into two 50 minutes segments.
Debugging is the often hated but yet nonetheless very important part of the developer's life fraught with lots of frustration and brief moments of glorious joy when the culprits are found. I am trying to give both the beginner and advanced audiences multiple techniques and tools to make the shining moments more frequent and frustrating times less so.
Here is my official description about the overall presentations:
Debugging can be a very frustrating exercise. Many times we find ourselves spending three times as long as it took us to develop the code to debug it. Especially code written by other developers seems time consuming to debug ;o)
Understanding the debugging infrastructure will shortcut much of the confusion and help to quickly focus on the source of the problem. What can you do when has reached is limits? We will talk about techniques, tools, and tips available to the ColdFusion Developer to make this a "less" frustrating exercise.
Hope to see you there,
B.
Debugging is the often hated but yet nonetheless very important part of the developer's life fraught with lots of frustration and brief moments of glorious joy when the culprits are found. I am trying to give both the beginner and advanced audiences multiple techniques and tools to make the shining moments more frequent and frustrating times less so.
Here is my official description about the overall presentations:
Debugging can be a very frustrating exercise. Many times we find ourselves spending three times as long as it took us to develop the code to debug it. Especially code written by other developers seems time consuming to debug ;o)
Understanding the debugging infrastructure will shortcut much of the confusion and help to quickly focus on the source of the problem. What can you do when
Hope to see you there,
B.
Subscribe to:
Posts (Atom)