<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Yet another web log</title>
	<atom:link href="http://blog.philippheckel.com/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.philippheckel.com</link>
	<description>Life, Linux and other things</description>
	<lastBuildDate>Thu, 17 Mar 2011 10:04:42 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3.1</generator>
		<item>
		<title>Hacking Flash Applications and Games: A Case Study</title>
		<link>http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/</link>
		<comments>http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#comments</comments>
		<pubDate>Tue, 01 Mar 2011 06:00:40 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Programming]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Flash]]></category>
		<category><![CDATA[Hacking]]></category>
		<category><![CDATA[SWF]]></category>
		<category><![CDATA[SWF Decompiler]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=674</guid>
		<description><![CDATA[Adobe&#8217;s proprietary Flash format has become a significant part of the Internet as it is today. While its importance in Web applications is slowly decreasing due to Web 2.0 technologies and HTML5, it is still a major player in the browser game and application sector. The majority of these games and applications are for entertainment [...]]]></description>
			<content:encoded><![CDATA[<p>Adobe&#8217;s proprietary Flash format has become a significant part of the Internet as it is today. While its importance in Web applications is slowly decreasing due to Web 2.0 technologies and HTML5, it is still a major player in the browser game and application sector. The majority of these games and applications are for entertainment only and offer you nothing but a nice rank in the high score. On rare occasions, however, there are Flash games in which you can actually win something valuable such as concert tickets or even money. And of course there are applications that let you stream, but not download content from a website.</p>
<p>So what&#8217;s the problem with that, you think? The problem is that almost <strong>all Flash applications can be hacked</strong> very easily and most developers are not aware of that. </p>
<p>As a reminder of how vulnerable Flash applications are, this post aims to raise awareness for these issues. In a case study, it shows how a Flash-based game and its server side high score can be tricked within a few minutes using free tools only.</p>
<p><span id="more-674"></span></p>
<hr />
<div class="toc">
<ol>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Disclaimer">Disclaimer</a></li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Introduction">1. Introduction</a>
<ol>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#The-problem-with-Flash-applications">1.1. The problem with Flash applications</a></li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Checksums-as-a-solution">1.2. Checksums as a solution?</a></li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Tools-and-basic-approach">1.3. Tools and basic approach</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Case-study-a-car-parking-game">2. Case study: a car parking game (with high score)</a>
<ol>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Capture-the-client-server-communication">2.1. Capture the client-server communication</a></li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Decompile-the-SWF-file-and-find-the-checksum-algorithm">2.2. Decompile the SWF file and find the checksum algorithm</a></li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Write-a-script">2.3. Write a script</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/#Conclusion">3. Conclusion</a></li>
</ol>
</div>
<hr />
<h3 id="Disclaimer" >Disclaimer</h3>
<p>This post is meant to raise awareness for the vulnerabilities of Flash-based applications. It is not a hacking tutorial or how-to. For that reason, I will not describe all steps in great detail, but only sketch the basic steps.</p>
<h3 id="Introduction" >1. Introduction</h3>
<h4 id="The-problem-with-Flash-applications" >1.1. The problem with Flash applications</h4>
<p>Flash applications are similar to normal Windows/Linux executables: Like normal programs, their source code is compiled to a binary format and later executed on the client machine. The major difference to .exe or Linux binaries is that decompiling is a lot easier: while a C or C++ compiler translates the source code into machine code, i.e. CPU instructions, SWF files contain the ActionScript code in plain text. That is with the right tools, <strong>extracting the complete code is a no-brainer</strong>!</p>
<p>Combining this with the fact that <strong>sniffing into the client-server communication</strong> is also not the most difficult task, one can easily simulate the Flash client with own code snippets and scripts, and thereby send <strong>forged requests</strong> to the server.</p>
<h4 id="Checksums-as-a-solution" >1.2. Checksums as a solution?</h4>
<p>Most application developers at least know that the client-server communication can be sniffed into. As an attempt to make it more difficult for people to send forged requests to the server, applications mostly use a message authentication code (MAC) or some other checksum-based mechanism. So instead of sending a plain text update query to the server (1), the application creates a hash by concatenating the user input with (to the user unknown) other data (2):</p>
<table style="width: 90%; margin: 5px auto; line-height: 130%" border="0">
<tr class="toprow">
<td style="width: 50%; text-align: left; font-size: 12px; padding: 5px">(1) Without checksum mechanism</td>
<td style="width: 50%; text-align: left; font-size: 12px; padding: 5px">(2) With checksum mechanism</td>
</tr>
<tr class="odd">
<td style='width: 50%; vertical-align: top; text-align: left; font-size: 12px; padding: 5px'>
<b>POST</b> /write-highscore.php <b>HTTP/1.1</b><br />
<b>Host</b>: www.example.com<br />
<b>User-Agent</b>: Mozilla/5.0 &#8230;<br />
&#8230;<br />
name=Phil&#038;score=40
</td>
<td style='width: 50%; vertical-align: top; text-align: left; font-size: 12px; padding: 5px'>
<b>POST</b> /write-highscore.php <b>HTTP/1.1</b><br />
<b>Host</b>: www.example.com<br />
<b>User-Agent</b>: Mozilla/5.0 &#8230;<br />
&#8230;<br />
name=Phil&amp;score=40&amp;<span style="color: red; font-weight: bold">check=283647a&#8230;</span>
</td>
</tr>
</table>
<p>This mechanism is completely based on the fact that the user does not know how the checksum is created. While this approach is very effective in regular applications, it completely fails in Flash programs: disassembling a Windows or Linux binary is very difficult and requires know how in assembler. Decompiling SWF files, however, is fairly easy and can be done in under a minute. That is retrieving the code that generates the checksum is only a matter of seconds.</p>
<h4 id="Tools-and-basic-approach" >1.3. Tools and basic approach</h4>
<p>The following steps present a generic approach on how any Flash application can be exploited like described above. </p>
<p><strong>Required Tools</strong>:</p>
<ul>
<li><a href="https://addons.mozilla.org/en-us/firefox/addon/firebug/">Firebug</a>: Firefox plugin for basic communication sniffing</li>
<li><a href="http://www.sothink.com/product/flashdecompiler/">Sothink Flash Decompiler</a>: required to extract ActionScript code (trial version is sufficient)</li>
<li><a href="http://www.wireshark.org/">Wireshark</a>: required for detailed communication sniffing (byte-for-byte)</li>
</ul>
<p><strong>Generic Approach</strong>:</p>
<ol>
<li>Enable Firebug and start sniffing with Wireshark.</li>
<li>Go to the target web site and play the game or use the application.</li>
<li>Look at the communication of the Flash application with the server. If there are any parts of the requests that cannot be recreated without the source code, e.g. checksums, download the important SWF files and decompile them.</li>
<li>Analyze the ActionScript code and find the part which creates the checksum.</li>
<li>Make a script that generates the same request with arbitrary input data, e.g. any score or name. For this script, the checksum algorithm from the ActionScript code can be used (or recreated).</li>
</ol>
<h3 id="Case-study-a-car-parking-game" >2. Case study: a car parking game (with high score)</h3>
<p>Our local newspaper <a href="http://www.morgenweb.de/">Mannheimer Morgen</a> recently hosted a competition in which users had to park a (virtual) car into different parking spots without damaging the car and with minimal fuel usage. Prizes were a safety training and concert tickets. While in this case the score of the participants did not decide who won (winners were drawn by lot), the winner in other competitions might be the one with the best score. <em>Please note that I did not take part in this competition and I did under no circumstances try to win the prizes.</em></p>
<p><img src="http://blog.philippheckel.com/wp-content/uploads/2011/02/parking-screenshot.png" alt="" title="parking-screenshot" width="450" height="349" class="aligncenter size-full wp-image-729" style="border: 0" /></p>
<p style="text-align: center; font-size: .8em; line-height: 115%"><strong>Parking Game</strong>: The target application in this case is a game called &#8220;Parking&#8221;.</p>
<h4 id="Capture-the-client-server-communication" >2.1. Capture the client-server communication</h4>
<p>Following the instructions from above, the first step is to capture the communication between the Flash client and the server. In this case, we are particularly interested in how to <strong>add our name with an arbitrary score to the high score list</strong>.</p>
<p><strong>Firebug</strong>:<br />
Using Firebug, we can see three interesting requests: </p>
<ul>
<li><em>savescore.swf</em> is the part of the client that is responsible for sending the score to the server.</li>
<li><em>serverdate-read.php</em> is called by savescore.swf for no obvious reason &#8211; very suspicious. We will later see that the result is used in the checksum generation.</li>
<li><em>highscore-write.php</em> is also called by savescore.swf and actually writes the user&#8217;s score to the database on the server.</li>
</ul>
<p><img src="http://blog.philippheckel.com/wp-content/uploads/2011/02/firebug.png" alt="" width="500" height="399" class="aligncenter size-full wp-image-735" /></p>
<p style="text-align: center; font-size: .8em; line-height: 115%"><strong>Firebug</strong> identifies the interesting files and requests.<br />In this case, three requests are relevant for the high score forgery.</p>
<p>What is particularly important is the two parameters <b>__ctrl</b> and <b>controlvalue</b>. Both are obviously generated by some checksum function in the <em>savescore.swf</em>-file. So the next step is to decompile the SWF file and look for the checksum-generating function.</p>
<p><strong>Wireshark</strong>:<br />
If we additionally enable Wireshark, we can get a plain text (or hex) representation of the HTTP request-response cycle. That is particularly important if one wants to forge a request in a way that it cannot be detected by the server. </p>
<h4 id="Decompile-the-SWF-file-and-find-the-checksum-algorithm" >2.2. Decompile the SWF file and find the checksum algorithm</h4>
<p>Using the SWF Decompiler software (trial version is sufficient), the <em>savescore.swf</em> can be examined very closely. Most of the applications are not very complex and the relevant code pieces are found very easily.</p>
<p><img src="/wp-content/uploads/2011/02/swfdecompile.png" alt="" width="500" height="334" class="aligncenter size-full wp-image-742" /></p>
<p>When analyzing <em>savescore.swf</em>, the two parts in which the checksums are created are found at different positions in the file. The <b>__ctrl</b> parameter is generated by taking a MD5 hash over the two concatenated values of <em>score</em> and <em>email</em>. The <b>controlvalue</b> parameter is created in a similar fashion and involves the previously queried <em>serverdate</em>.</p>
<p><img src="/wp-content/uploads/2011/02/checksum1.png" alt="" width="491" height="197" class="aligncenter size-full wp-image-740" /></p>
<p style="text-align: center; font-size: .8em; line-height: 115%">The <b>__ctrl</b> parameter is a hash value over two of the variable input parameters.<br />The <b>controlvalue</b> parameter is created similarly.</p>
<h4 id="Write-a-script" >2.3. Write a script</h4>
<p>Once it is clear how the checksums are generated, the puzzle is solved. The only thing left is to write a script that allows entering arbitrary input values, and using the previously determined checksum-mechanisms to generate a MAC.</p>
<p>Here is a short excerpt of how this could look like (using PHP):</p>

<div class="wp_syntax"><div class="code"><pre class="php" style="font-family:monospace;"><span style="color: #000088;">$score</span> <span style="color: #339933;">=</span> <span style="color: #0000ff;">&quot;999.999&quot;</span><span style="color: #339933;">;</span>
<span style="color: #000088;">$email</span> <span style="color: #339933;">=</span> <span style="color: #0000ff;">&quot;nobody@example.com&quot;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000088;">$data</span> <span style="color: #339933;">=</span> <span style="color: #990000;">array</span><span style="color: #009900;">&#40;</span>
	<span style="color: #0000ff;">&quot;__ctrl&quot;</span> <span style="color: #339933;">=&gt;</span> <span style="color: #990000;">md5</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$score</span><span style="color: #339933;">.</span><span style="color: #000088;">$email</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span> 
	<span style="color: #339933;">...</span>
	<span style="color: #0000ff;">&quot;score&quot;</span> <span style="color: #339933;">=&gt;</span> <span style="color: #000088;">$score</span><span style="color: #339933;">,</span>
	<span style="color: #0000ff;">&quot;controlvalue&quot;</span> <span style="color: #339933;">=&gt;</span> <span style="color: #990000;">md5</span><span style="color: #009900;">&#40;</span><span style="color: #339933;">...</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">,</span> 
	<span style="color: #339933;">...</span>
<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #b1b100;">foreach</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$data</span> <span style="color: #b1b100;">as</span> <span style="color: #000088;">$k</span><span style="color: #339933;">=&gt;</span><span style="color: #000088;">$v</span><span style="color: #009900;">&#41;</span> 
	<span style="color: #000088;">$data_enc</span><span style="color: #009900;">&#91;</span><span style="color: #009900;">&#93;</span> <span style="color: #339933;">=</span> <span style="color: #990000;">urlencode</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$k</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">.</span><span style="color: #0000ff;">&quot;=&quot;</span><span style="color: #339933;">.</span><span style="color: #990000;">urlencode</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$v</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000088;">$req</span> <span style="color: #339933;">=</span> <span style="color: #0000ff;">&quot;POST /parking/highscore-write.php HTTP/1.1<span style="color: #000099; font-weight: bold;">\r</span><span style="color: #000099; font-weight: bold;">\n</span>&quot;</span>
     <span style="color: #339933;">.</span> <span style="color: #0000ff;">&quot;Host: www.morgenweb.de<span style="color: #000099; font-weight: bold;">\r</span><span style="color: #000099; font-weight: bold;">\n</span>&quot;</span>
     <span style="color: #339933;">...</span>
     <span style="color: #339933;">.</span> <span style="color: #990000;">join</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;&amp;&quot;</span><span style="color: #339933;">,</span><span style="color: #000088;">$data_enc</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #000088;">$fp</span> <span style="color: #339933;">=</span> <span style="color: #990000;">fsockopen</span><span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;www.morgenweb.de&quot;</span><span style="color: #339933;">,</span> <span style="color: #cc66cc;">80</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #990000;">fwrite</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$fp</span><span style="color: #339933;">,</span> <span style="color: #000088;">$req</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
<span style="color: #b1b100;">while</span> <span style="color: #009900;">&#40;</span><span style="color: #000088;">$line</span> <span style="color: #339933;">=</span> <span style="color: #990000;">fgets</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$fp</span><span style="color: #009900;">&#41;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span> <span style="color: #009900;">&#125;</span>	
<span style="color: #990000;">fclose</span><span style="color: #009900;">&#40;</span><span style="color: #000088;">$fp</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span></pre></div></div>

<h3 id="Conclusion" >3. Conclusion</h3>
<p>This blog post introduced a common way to outsmart Flash applications and games. It demonstrated that by sniffing into client-server communication and decompiling SWF files, many Flash applications can be misused. Using the example of a Flash game, the post showed that SWF cannot be compared to Windows/Linux binaries, but must rather be seen as JavaScript-like client code. </p>
<p>Even though this post only demonstrated this using a rather harmless game, more serious misuse is also possible. Flash-based music streaming sites such as <a href="http://www.simfy.com/">Simfy</a>, <a href="http://www.spotify.com/">Spotify</a> <a href="http://www.grooveshark.com/">Grooveshark</a> are affected as well. They all use a similar mechanism for their Flash-based music player. Using the approach presented above combined with tools like <em>rtmpdump</em> can potentially harm their services significantly.</p>
<p>Flash once was a great way for bringing a little dynamic in the Web 1.0, but is no outdated. With various JavaScript frameworks and HTML5 on the way, Flash is going to retire soon. Developers and companies must be aware of the flaws of Flash and adjust their services accordingly to make sure that they cannot be misused.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2011/03/01/hacking-flash-games-a-case-study/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Altering old SVN revisions: removing confidental data from a Subversion repository</title>
		<link>http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/</link>
		<comments>http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#comments</comments>
		<pubDate>Tue, 01 Feb 2011 11:26:44 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Administration]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Subversion]]></category>
		<category><![CDATA[SVN]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=588</guid>
		<description><![CDATA[Version control systems like CVS or Subversion are designed for keeping track of the changes of a project and for having the possibility to revert to old revisions if something goes wrong. In contrast to regular relational databases, these systems are made only for adding new content to a repository, and not for removing data [...]]]></description>
			<content:encoded><![CDATA[<p>Version control systems like <a href="http://www.nongnu.org/cvs/">CVS</a> or <a href="http://subversion.apache.org/">Subversion</a> are designed for keeping track of the changes of a project and for having the possibility to revert to old revisions if something goes wrong. In contrast to regular relational databases, these systems are made only for adding new content to a repository, and not for removing data from it. In fact, deleting old content is <a href=" http://subversion.apache.org/faq.html#removal">not a built-in functionality</a> in SVN, and mostly requires removing entire revisions from the repository or even <a href="http://svnbook.red-bean.com/en/1.1/ch05s03.html#svn-ch-5-sect-3.1.3">creating a new one</a>. </p>
<p>But what happens if you <strong>accidentally commit a password</strong> or other sensitive information to a repository? This post explains <strong>how to remove this confidential data</strong> permanently from the repository by simply overwriting it in old revisions, i.e. without having to remove revisions or create a new repository.</p>
<p><span id="more-588"></span></p>
<hr />
<b>Contents</b></p>
<div class="toc">
<ol>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Introduction">1. Introduction</a></p>
<ol>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Disclaimer">1.1. Disclaimer</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Requirements">1.2. Requirements</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Example-Scenario">1.3. Example Scenario</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Local-machine-Identify-the-affected-revisions-in-the-working-copy">2. Local machine: Identify the affected revisions in the working copy</a>
<ol>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Fix-and-commit-the-affected-file">2.1. Fix and commit the affected file</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Identify-the-affected-file-versions-locally">2.2. Identify the affected file versions locally</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Get-MD5-checksums-of-the-affected-versions">2.3. Get MD5 checksums of the affected versions</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#SVN-repository-Correct-the-affected-files">3. SVN repository: Correct the affected files</a>
<ol>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Make-a-repository-backup">3.1. Make a repository backup</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Verify-affected-versions">3.2. Verify affected versions</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Replace-the-password-and-checksums">3.3. Replace the password and checksums</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Test-locally">4. Test locally</a></li>
<li><a href="http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/#Bash-history-cleanup">5. Bash history cleanup</a></li>
</ol>
</div>
<hr />
<h3 id="Introduction" >1. Introduction</h3>
<h4 id="Disclaimer" >1.1. Disclaimer</h4>
<p>The following actions might lead to data loss. I am not responsible for anything that goes wrong because of my description. </p>
<h4 id="Requirements" >1.2. Requirements</h4>
<p>It is absolutely necessary to have <strong>root access to the SVN respository</strong>. That is not only through the svnadmin command, but full command line access to the files, particularly to the &#8220;repos&#8221; directory.</p>
<p>If you do not have root access to the repository, you cannot remove any data from the repository! In that case, contact your SVN administrator.</p>
<h4 id="Example-Scenario" >1.3. Example Scenario</h4>
<p>For this example, let&#8217;s assume you accidentally committed the file <strong>config.cfg</strong> with a plain text password <em style="color: red">123secret</em> a while ago (in <strong>revision 12</strong>). The repository is currently at <strong>revision 25</strong> and you just realized that the password was in there all the time:</p>

<div class="wp_syntax"><div class="code"><pre class="config" style="font-family:monospace;"># Config file &quot;config.cfg&quot;
username = someone
password = 123secret
...</pre></div></div>

<h3 id="Local-machine-Identify-the-affected-revisions-in-the-working-copy" >2. Local machine: Identify the affected revisions in the working copy</h3>
<h4 id="Fix-and-commit-the-affected-file" >2.1. Fix and commit the affected file</h4>
<p>The following commands are performed on <strong>your local machine</strong> within the working copy of the project, i.e. on the client machine.</p>
<p>Before we start tinkering and forging the SVN history and its repository, first fix the affected file and commit a new revision to the repository. In most cases, people are not going to look in old revisions of a config file, so the faster you commit a new version, the less likely it is that someone sees it!</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #7a0874; font-weight: bold;">cd</span> ~<span style="color: #000000; font-weight: bold;">/</span>Dev<span style="color: #000000; font-weight: bold;">/</span>yourproject
$ <span style="color: #c20cb9; font-weight: bold;">vi</span> config.cfg
<span style="color: #666666; font-style: italic;"># Change password to something else</span>
$ <span style="color: #c20cb9; font-weight: bold;">svn</span> commit <span style="color: #660033;">-m</span> <span style="color: #ff0000;">&quot;config update&quot;</span>
...
Transmitting <span style="color: #c20cb9; font-weight: bold;">file</span> data .
Committed revision <span style="color: #000000;">26</span>.</pre></div></div>

<h4 id="Identify-the-affected-file-versions-locally" >2.2. Identify the affected file versions locally</h4>
<p>In most cases you will probably realize right away that you just committed something confidential to the SVN repository. In this case, you only have to fix one single version of that file and is pretty clear which revision is affected.</p>
<p>In other cases, however, the affected file might be in the repository for many revisions before you realize it. If this is the case, there might be multiple revisions of the file in the repository and each of these versions needs to be fixed. To identify the possibly affected versions of the file, you can peak into the logs:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">svn</span> log config.cfg
<span style="color: #660033;">------------------------------------------------------------------------</span>
r22 <span style="color: #000000; font-weight: bold;">|</span> someone <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #000000;">2011</span>-01-<span style="color: #000000;">25</span> 01:<span style="color: #000000;">36</span>:<span style="color: #000000;">12</span> +0100 <span style="color: #7a0874; font-weight: bold;">&#40;</span>Tue, <span style="color: #000000;">25</span> Jan <span style="color: #000000;">2011</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #000000;">1</span> line
&nbsp;
update xy config
<span style="color: #660033;">------------------------------------------------------------------------</span>
r12 <span style="color: #000000; font-weight: bold;">|</span> someone <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #000000;">2011</span>-01-05 00:<span style="color: #000000;">45</span>:<span style="color: #000000;">19</span> +0100 <span style="color: #7a0874; font-weight: bold;">&#40;</span>Wed, 05 Jan <span style="color: #000000;">2011</span><span style="color: #7a0874; font-weight: bold;">&#41;</span> <span style="color: #000000; font-weight: bold;">|</span> <span style="color: #000000;">1</span> line
&nbsp;
added connection details to config
<span style="color: #660033;">------------------------------------------------------------------------</span>
...</pre></div></div>

<p>In this case, the file has been altered in the two <strong>revisions 12 and 22</strong>. Both <strong>might</strong> include the password and are stored in the repository, i.e. both potentially need to be corrected.</p>
<h4 id="Get-MD5-checksums-of-the-affected-versions" >2.3. Get MD5 checksums of the affected versions</h4>
<p>SVN ensures the integrity of its repository by saving MD5 checksums of all the files and its versions. Since it is now clear which revisions might be affected, you need to get the <em>current</em> checksums of these file versions and calculate checksums for the new <em>corrected</em> (&#8220;forged&#8221;) versions. In short, you need to do the following <strong>for each affected version</strong>:</p>
<ul>
<li>Retrieve the version and calculate its MD5 checksum</li>
<li>Make a copy of file, replace the confidential information with &#8220;x&#8221;s and calculate the MD5 checksum of the new file.</li>
<li>Remember or copy all the checksums and versions into a file.</li>
</ul>
<p>In this example, we&#8217;ll have to get the checksums for revisions 12 and 22 of the <em>config.cfg</em>-file. The code below only shows what to do for revision 22; revision 12 is analogue:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;"># Get current checksum of revision 22</span>
$ <span style="color: #c20cb9; font-weight: bold;">svn</span> <span style="color: #660033;">--revision</span> <span style="color: #000000;">22</span> config.cfg
...
At revision <span style="color: #000000;">22</span>.
$ md5sum config.cfg
0e28c6c8342649c290400567130f657b  config.cfg
&nbsp;
<span style="color: #666666; font-style: italic;"># Correct the file and get new checksum</span>
$ <span style="color: #c20cb9; font-weight: bold;">cp</span> config.cfg <span style="color: #000000; font-weight: bold;">/</span>tmp<span style="color: #000000; font-weight: bold;">/</span>config.cfg-<span style="color: #000000;">22</span>
$ <span style="color: #c20cb9; font-weight: bold;">vi</span> <span style="color: #000000; font-weight: bold;">/</span>tmp<span style="color: #000000; font-weight: bold;">/</span>config.cfg-<span style="color: #000000;">22</span>
<span style="color: #666666; font-style: italic;"># Overwrite the password with &quot;xxxxxxxxx&quot; (same length as the old password!!)</span>
$ md5sum <span style="color: #000000; font-weight: bold;">/</span>tmp<span style="color: #000000; font-weight: bold;">/</span>config.cfg-<span style="color: #000000;">22</span>
459a78e2eae02b28f810f9fdebdc5b52  <span style="color: #000000; font-weight: bold;">/</span>tmp<span style="color: #000000; font-weight: bold;">/</span>config.cfg-<span style="color: #000000;">22</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Repeat this for revision 12</span></pre></div></div>

<h3 id="SVN-repository-Correct-the-affected-files" >3. SVN repository: Correct the affected files</h3>
<div style="border: 1px solid red; background: #fcc; padding: 5px; margin-top: 8px"><strong>Warning</strong>: This step can damage your repository, so make sure you backup as described in 3.1. before you change anything.</div>
<p>In this step, we finally start altering the repository. All the actions are performed on <strong>the server machine</strong> as <strong>root</strong> user inside the actual SVN repository directory, so be sure not to confuse it with you local machine.</p>
<h4 id="Make-a-repository-backup" >3.1. Make a repository backup</h4>
<p>Creating some sort of backup is crucial, since we are about to change the binary revision files of the Subversion repository. The easiest way to do this is to backup the whole repository folder of your project, e.g. /path/to/svn/repos/yourproject. However, if its total size is too big you can also choose to only backup the files identified in 3.2.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #666666; font-style: italic;"># Make backup of project; Note the &quot;-a&quot; parameter to keep the permissions.</span>
$ <span style="color: #c20cb9; font-weight: bold;">mkdir</span> <span style="color: #000000; font-weight: bold;">/</span>backups
$ <span style="color: #c20cb9; font-weight: bold;">cp</span> <span style="color: #660033;">-a</span> <span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>svn<span style="color: #000000; font-weight: bold;">/</span>repos<span style="color: #000000; font-weight: bold;">/</span>yourproject <span style="color: #000000; font-weight: bold;">/</span>backups</pre></div></div>

<h4 id="Verify-affected-versions" >3.2. Verify affected versions</h4>
<p>After the backup, we need to verify that we really need to change all the versions we identified earlier. To do that, navigate to the &#8220;revs&#8221; folder inside the repository and grep for the password:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #7a0874; font-weight: bold;">cd</span> <span style="color: #000000; font-weight: bold;">/</span>path<span style="color: #000000; font-weight: bold;">/</span>to<span style="color: #000000; font-weight: bold;">/</span>svn<span style="color: #000000; font-weight: bold;">/</span>repos<span style="color: #000000; font-weight: bold;">/</span>yourproject<span style="color: #000000; font-weight: bold;">/</span>db<span style="color: #000000; font-weight: bold;">/</span>revs<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">0</span>
$ <span style="color: #c20cb9; font-weight: bold;">grep</span> 123secret <span style="color: #000000; font-weight: bold;">*</span>
Binary <span style="color: #c20cb9; font-weight: bold;">file</span> <span style="color: #000000;">12</span> matches
Binary <span style="color: #c20cb9; font-weight: bold;">file</span> <span style="color: #000000;">22</span> matches</pre></div></div>

<p>The matching files are the revisions that contain the password, and hence also the files that need to be &#8220;corrected&#8221;. Note that sometimes not all the versions identified through the &#8220;svn log&#8221; command appear in this list. That is because when the file is simply moved and not changed or other parts of it were altered, its contents will not be stored in the SVN revision file.</p>
<h4 id="Replace-the-password-and-checksums" >3.3. Replace the password and checksums</h4>
<p>Since the SVN revision files are binary, we need a hex editor to edit them. Hence install hexedit, and then simply replace the password and checksums like identified before:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> hexedit
&nbsp;
$ hexedit <span style="color: #000000;">22</span>
<span style="color: #666666; font-style: italic;"># Hex editor opens the file for revision 22</span>
<span style="color: #666666; font-style: italic;"># Replace passwords and checksums</span>
&nbsp;
<span style="color: #666666; font-style: italic;"># Repeat this for revision 12</span></pre></div></div>

<p>Hexedit is not the easiest editor to use. So here is a step-by-step of what you need to do:</p>
<ul>
<li>Hit TAB, then CTRL-S to search</li>
<li>Enter the password <em style="color: red">123secret</em> and hit return</li>
<li>Overwrite the password with <em style="color: red">xxxxxxxxx</em> (same length!)</li>
<li>Hit CTRL-S, then &#8220;Y&#8221; to save</li>
<li>Repeat 1-4 for each occurance of the password.</li>
<li>Do the same for the old checksum &#8220;0e28c6c8342649c290400567130f657b&#8221;, and replace it with the new one &#8220;f85abfd8b63fa7ab68abc9364f2d339e&#8221;</li>
<li>Hit CTRL-X to quit</li>
<li>Repeat this for all affected revisions</li>
</ul>
<p>That&#8217;s the complete magic. If checked out, the revisions 12 and 22 (and of course also their succeeding versions) will show <em style="color: red">xxxxxxxxx</em> instead of the initially committed password.</p>
<h3 id="Test-locally" ">4. Test locally</h3>
<p>Now test locally if you can switch between revisions and every works without error messages:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">svn</span> <span style="color: #660033;">--revision</span> <span style="color: #000000;">12</span> update
...
At revision <span style="color: #000000;">12</span>.
$ <span style="color: #c20cb9; font-weight: bold;">grep</span> password config.cfg
password = xxxxxxxxx</pre></div></div>

<p>If you did everything as the tutorial says, you shouldn&#8217;t get any errors. If you forgot to replace checksums or you changed something that you weren&#8217;t supposed to change in the SVN revision file, you might get an error like below. However, if that happens, you can always go back to your backup and try it again&#8230;</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">svn</span> <span style="color: #660033;">--revision</span> <span style="color: #000000;">12</span> update
<span style="color: #c20cb9; font-weight: bold;">svn</span>: Checksum mismatch <span style="color: #000000; font-weight: bold;">while</span> reading representation:
   expected:  f85abfd8b63fa7ab68abc9364f2d339e
     actual:  de6f581d115197baebc43c3975b9e396</pre></div></div>

<h3 id="Bash-history-cleanup" >5. Bash history cleanup</h3>
<p>In step 3.2. we typed the plain text password in the bash. As you might know, this leaves traces in the ~/.bash_history file. Delete them by opening the files and then by simply removing the according lines. Make sure that you <strong>do not use the search</strong> function of VIM, since that has a history on its own. If you do, delete the history of VIM in ~/.viminfo.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">vi</span> ~<span style="color: #000000; font-weight: bold;">/</span>.bash_history
<span style="color: #666666; font-style: italic;"># Remove everything that contains the password</span>
<span style="color: #666666; font-style: italic;"># Do NOT use the search function, but search manually!</span></pre></div></div>

]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2011/02/01/altering-old-svn-revisions-removing-confidental-data-from-subversion-repository/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Picasa for Linux: Export albums in sort order</title>
		<link>http://blog.philippheckel.com/2011/01/12/picasa-for-linux-export-pictures-in-sort-order/</link>
		<comments>http://blog.philippheckel.com/2011/01/12/picasa-for-linux-export-pictures-in-sort-order/#comments</comments>
		<pubDate>Wed, 12 Jan 2011 19:00:47 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Multimedia]]></category>
		<category><![CDATA[Perl]]></category>
		<category><![CDATA[Picasa]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=568</guid>
		<description><![CDATA[As one of the best picture organizers out there, Picasa is (in my opinion) almost complete in terms of features and has a nice look and feel at the same time. Even though Google stopped developing the Linux version after 3.0, it still works perfectly using Wine and a couple of cp-statements. However, as stated [...]]]></description>
			<content:encoded><![CDATA[<p>As one of the best picture organizers out there, <a href="http://picasa.google.com/index.html##">Picasa</a> is (in my opinion) almost complete in terms of features and has a nice look and feel at the same time. Even though Google stopped developing the <a href="http://picasa.google.com/linux/">Linux version</a> after 3.0, it still works perfectly using Wine and <a href="http://www.webupd8.org/2010/04/how-to-install-picasa-36-in-ubuntu.html">a couple of cp-statements</a>.</p>
<p>However, as stated many times by Picasa users and bloggers [<a href="http://blog.alex-kunz.de/2009/01/picasa-exporting-albums-correctly.html">1</a>,<a href="http://www.google.com/support/forum/p/Picasa/thread?tid=46f5680a3047ae21&#038;hl=en">2</a>,<a href="http://www.google.com/support/forum/p/Picasa/thread?tid=6290b10dc34e6de7&#038;hl=en">3</a>,<a href="http://www.google.com/support/forum/p/Picasa/thread?tid=38a72e56f77f06a6&#038;hl=en">4</a>,...], Picasa&#8217;s export function misses a tiny little feature that maintains the sort order of the album when exporting it to a folder. Instead of renaming the pictures to keep them sorted in normal file managers (by name), Picasa just copies the files of an album to one folder and thereby destroys the order. As if that wasn&#8217;t enough, Picasa also overwrites duplicates filenames from different source folders. </p>
<p>This missing feature has even led to small standalone projects that fix this issue, e.g. <a href="http://code.google.com/p/piae/">Picasa Independent Album Exporter (PIAE)</a> and <a href="http://sourceforge.net/projects/picasaorderpres/">Picasa Order Preserver</a>. While both applications do their job, both are a bit heavyweight, and PIAE only works for Windows (and not on Wine). </p>
<p>This post presents a tiny little Perl script that renames pictures of an exported album according to their Picasa sort order.</p>
<p><span id="more-568"></span></p>
<h3 id="toc-download-installation">Download &amp; Installation</h3>
<p>The script requires Perl module <a href="http://search.cpan.org/dist/XML-Simple/">XML::Simple</a> to be installed. In Debian/Ubuntu, install the package <a href="apt://libxml-simple-perl">libxml-simple-perl</a>.</p>
<p>Download the script, save it to your preferred location and make it executable:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> libxml-simple-perl
$ <span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">wget</span> <span style="color: #660033;">-O</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>local<span style="color: #000000; font-weight: bold;">/</span>bin<span style="color: #000000; font-weight: bold;">/</span>picasa-export \
          http:<span style="color: #000000; font-weight: bold;">//</span>blog.philippheckel.com<span style="color: #000000; font-weight: bold;">/</span>uploads<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">2011</span><span style="color: #000000; font-weight: bold;">/</span>01<span style="color: #000000; font-weight: bold;">/</span>picasa-export
$ <span style="color: #c20cb9; font-weight: bold;">sudo</span> <span style="color: #c20cb9; font-weight: bold;">chmod</span> +x <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>local<span style="color: #000000; font-weight: bold;">/</span>bin<span style="color: #000000; font-weight: bold;">/</span>picasa-export</pre></div></div>

<p>That&#8217;s it for the installation.</p>
<p><strong>Download</strong>: <a href="http://blog.philippheckel.com/uploads/2011/01/picasa-export">Picasa Export 0.1, January 2011</a></p>
<h3 id="toc-usage">Usage</h3>
<p>Using the script is also very simple. Simple export the desired album using Picasa&#8217;s &#8220;Export to HTML Page&#8221; function and call the script with the exported folder as parameter.</p>
<p><strong>Example with &#8220;YourAlbum&#8221;:</strong></p>
<ol>
<li><strong>Picasa</strong>: Click &#8220;Folder&#8221; → &#8220;Export to HTML Page&#8230;&#8221;. Select &#8220;XML Code&#8221; in the template list. Click &#8220;Export&#8221;.</li>
<li><strong>Console</strong>:

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ picasa-export ~<span style="color: #000000; font-weight: bold;">/</span><span style="color: #ff0000;">&quot;Picasa\ HTML\ Exports/YourAlbum&quot;</span></pre></div></div>

</li>
</ol>
<p>After that, the new folder &#8220;neworder&#8221; contains the renamed pictures.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2011/01/12/picasa-for-linux-export-pictures-in-sort-order/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
		</item>
		<item>
		<title>Aware Context API (ACAPI) &#8211; Using the sensors in Java ME</title>
		<link>http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/</link>
		<comments>http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#comments</comments>
		<pubDate>Sun, 22 Aug 2010 05:43:33 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Mobile]]></category>
		<category><![CDATA[Programming]]></category>
		<category><![CDATA[Java]]></category>
		<category><![CDATA[Java ME]]></category>
		<category><![CDATA[Symbian]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=481</guid>
		<description><![CDATA[The capabilities of cell phones increased dramatically in the last few years. While in the old days most mobile phones were primarily used to make phone calls, modern smart-phones are mostly all-round devices. With the possibility of accessing the Internet and the availability of various sensors (e.g. location or noise), mobile applications have become interactive [...]]]></description>
			<content:encoded><![CDATA[<p>The capabilities of cell phones increased dramatically in the last few years. While in the old days most mobile phones were primarily used to make phone calls, modern smart-phones are mostly all-round devices. With the possibility of accessing the Internet and the availability of various sensors (e.g. location or noise), mobile applications have become interactive and flexible. The trend towards location-based services and context-awareness allows applications to react on their surroundings and to behave intuitively towards the user.</p>
<p>For developers, context-awareness can be both a blessing and a curse. While the mobile operating systems iPhone OS and Android come with relatively good sensor-support, the vast majority of devices deal with Java ME&#8217;s basic and heterogeneous sensor functionalities.</p>
<p>The <strong>Aware Context API (ACAPI)</strong> aims to bridge this gap by providing a framework for building context aware applications for mobile devices based on Java ME. In this article, I&#8217;d like to introduce ACAPI, its structure and usage briefly. Please feel free to comment.</p>
<p><span id="more-481"></span></p>
<hr />
<b>Contents</b></p>
<div class="toc">
<ol>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Motivation-and-Goals">1. Motivation and Goals</a></li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Development-Team-and-Scope">2. Development Team and Scope</a></li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Aware-Context-API-ACAPI">3. Aware Context API (ACAPI)</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#ACAPI-Structure">3.1. ACAPI Structure</a></li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Implemented-Sensors">3.2. Implemented Sensors</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Example-Usage">4. Example Usage</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Example-Scenario-and-Code">4.1. Example Scenario and Code</a></li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Proof-of-Concept-Application">4.2. Proof-of-Concept Application</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Future-Work-and-Conclusion">5. Future Work and Conclusion</a></li>
<li><a href="http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/#Download-and-License">A. Download and License</a></li>
</ol>
</div>
<hr />
<h3 id="Motivation-and-Goals" >1. Motivation and Goals</h3>
<p>ACAPI is designed to allow easy and homogeneous access to the different sensors of the mobile device. It creates an abstraction between the available sensors and their usage so that developers can focus on the business logic rather than on how to use the sensor implementations.</p>
<blockquote><p><em>Example:</em> A mobile application shall notify the user if another (previously defined) person comes into his or her range, e.g. if the boss arrives at the office.</p></blockquote>
<p>Using the standard Java ME interfaces, developers have to get to know the different APIs and write a lot of code to solve this or similar problems. In this use case, the application needs to determine its position (<em>the office</em>), monitor the available devices around it (<em>the phone of the boss</em>) and be able to notify the user when both events occur. </p>
<p>The Aware Context API aims to solve this reoccurring problem with an easy-to-use event-based framework that allows defining <em>rules</em> for available sensor data. Using ACAPI, the given example can be solved easily by defining a <em>rule</em> and an <em>action</em> that is triggered when the rule matches:</p>
<ul>
<li><strong>Rule</strong>: Wi-Fi &#8220;OfficeWLAN&#8221; available <em>AND</em> Bluetooth device &#8220;BossPhone&#8221; available</li>
<li><strong>Action</strong>: Notify user, e.g. by playing a sound</li>
</ul>
<h3 id="Development-Team-and-Scope" >2. Development Team and Scope</h3>
<p>The ACAPI project was developed within the scope of a team project as part of the curriculum of a Master of Science Degree at the <a href="http://www.uni-mannheim.de/">University of Mannheim</a>. The project was a team-effort, carried out over a 1-year duration at the <a href="http://wifo1.bwl.uni-mannheim.de/">Chair of Business Administration and Information Systems</a>, under the supervision of <a href="http://wifo1.bwl.uni-mannheim.de/heinzl.html">Prof. Armin Heinzl</a> and his research assistants <a href="http://wifo1.bwl.uni-mannheim.de/hemmer.html">Erik Hemmer</a> and <a href="http://wifo1.bwl.uni-mannheim.de/stuckenberg.html">Sebastian Stuckenberg</a>. </p>
<p>The project team consisted of <a href="https://www.xing.com/profile/Lars_Bakker">Lars Bakker</a>, <a href="https://www.xing.com/profile/Philipp_Heckel">Philipp Heckel</a> (myself), <a href="https://www.xing.com/profile/Obie_Modisane">Obie Modisane</a>, <a href="https://www.xing.com/profile/Benjamin_Schubert13">Benjamin Schubert</a> and <a href="https://www.xing.com/profile/Moritz_Waechter2">Moritz Wächter</a>.</p>
<h3 id="Aware-Context-API-ACAPI" >3. Aware Context API (ACAPI)</h3>
<p>The Aware Context API is well-structured and is very easy to understand. It is easily extendible and supports a broad range of devices. It is mainly based on Java ME, but has native parts whenever needed (e.g. for Wi-Fi, battery or telephony). </p>
<h4 id="ACAPI-Structure" >3.1. ACAPI Structure</h4>
<p>ACAPI is horizontally structured into 3 different layers:</p>
<ul>
<li><strong>Sensor</strong>: On the lowest level, the <em>sensors</em> gather data about the current status and context of the phone. A Wi-Fi sensor, for instance, collects available devices and it issues an event whenever the data changes. Applications can either hook themselves directly into the sensor events or use higher abstractions (conditions and rules).</li>
<li><strong>Condition</strong>: In order to evaluate a single sensor, <em>conditions</em> compare the sensor&#8217;s properties to given values. They can become either <em>true</em> or <em>false</em>. A location condition, for example, becomes <em>true</em> if the phone gets into the range of certain coordinates. Similar to a sensor, a condition issues an event when the value changes (from <em>true</em> to <em>false</em>, or vice versa).</li>
<li><strong>Rule</strong>: To express more than one condition, <em>rules</em> can combine conditions to a more complex logical expression. In the above example, the rule only matches if both conditions match (&#8220;in the office&#8221; and &#8220;boss phone available&#8221;).</li>
</ul>
<p><img src="http://blog.philippheckel.com/wp-content/uploads/2010/08/acapi-structure.png" alt="" title="ACAPI structure" class="aligncenter size-full wp-image-503" style="border: none" /></p>
<p style="font-size: .8em; line-height: 115%"><strong>ACAPI Structure</strong> (simplified and incomplete!): The Aware Context API is layer-based. Each of the components is easily extendible and has event listeners to react on changes. This chart shows the interdependence of the different layers.</p>
<p>Using this layered structure, ACAPI fundamentally changes the development strategy of mobile applications. Instead of predefining a screen and/or process flow, applications are <strong>event-driven</strong>. Whenever a rule changes its state (match vs. no match), the application can react by displaying a different screen, notifying the user, or by performing other actions.</p>
<p>Besides the horizontal division, the API is also vertically divided in the two logical parts <em>Boolean</em> and <em>Fuzzy</em>. While the Boolean part assumes correct sensors, the Fuzzy conditions and rules take measurement errors and inaccuracy into account. While the Boolean conditions and rules can only become <em>true</em> or <em>false</em>, their Fuzzy counterparts implement a score-based system that only triggers when a certain threshold is reached. This is particularly relevant for sensors that supply accuracy data, e.g. GPS sensors.</p>
<h4 id="Implemented-Sensors" >3.2. Implemented Sensors</h4>
<p>The current code base of ACAPI includes several predefined sensors, including the most common: Bluetooth and GPS. Most sensors are entirely based on Java ME and will work on any phone that supports the corresponding JSR. However, since Java ME does not provide access to some functionalities, a few native implementations are required (e.g. for Wi-Fi, battery status or telephony status). </p>
<p>The following sensors are already implemented:</p>
<ul>
<li><strong>Battery Sensor</strong> (native S60): This sensor monitors the status of the battery (%) and the charger (enum value, e.g. on-battery, or plugged-in). There is currently only a Symbian S60 implementation for this sensor since Java ME does not allow access to the battery data.</li>
<li><strong>Bluetooth Sensor</strong>: This sensor monitors available devices, e.g. phones or laptops. It can react on joining or leaving devices. </li>
<li><strong>Custom Sensor</strong>: This sensor allows the integration of business logic in ACAPI, so that rules do not only include actual sensor data, but also virtual business sensor data.</li>
<li><strong>Location Sensor</strong> (GPS and Wi-Fi; partially native S60): This sensor monitors the position and the speed. It uses GPS and Wi-Fi triangulation to get a fast and accurate position. Since Java ME does not allow access to the wireless sensor, the Wi-Fi part is native S60 code.</li>
<li><strong>Noise Level  Sensor</strong>: This sensor monitors the noise level (<em>in decibels</em>) of the surrounding area.</li>
<li><strong>Time Sensor</strong>: This sensor delivers the current time and can react on date and time changes.</li>
<li><strong>Wireless Sensor</strong> (native S60): This sensor monitors the available Wi-Fi devices, i.e. access points. It can react on joining and leaving devices. In combination with a web service, it can be used to estimate the position. </li>
</ul>
<p>There are many other possible sensors that could be implemented using the available abstract classes. Examples include an orientation sensor (<em>react on device movement</em>) or a telephony sensor (<em>react on calls, SMS etc.</em>).</p>
<h3 id="Example-Usage" >4. Example Usage</h3>
<p>Having discussed the structure of the Aware Context API, the following section elaborates the above-mentioned example even further. It explains the scenario and shows specific example code.</p>
<h4 id="Example-Scenario-and-Code" >4.1. Example Scenario and Code</h4>
<p>As already briefly mentioned above, the example scenario for demonstrating the API is very simple: The application shall display a warning message and play a warning sound when the boss arrives at the office. </p>
<p><img src="/wp-content/uploads/2010/08/acapi-usecase.png" alt="" title="Use Case" width="500" height="205" class="aligncenter size-full wp-image-493" style="border: none" /></p>
<p>The two conditions depicted in the diagram above are combined in one Boolean rule, i.e. the rule only becomes <em>true</em> if both of the conditions match.</p>
<p>Similar to the API concept, its actual usage is also very simple. The following code snippet shows how to implement the above example in a regular Java ME application.</p>

<div class="wp_syntax"><table><tr><td class="line_numbers"><pre>1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
</pre></td><td class="code"><pre class="java" style="font-family:monospace;"><span style="color: #666666; font-style: italic;">// Conditions</span>
Condition inOffice <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> WirelessNearCondition<span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;OfficeWLAN&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
Condition bossPhone <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> BluetoothNearCondition<span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;BossPhone&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #666666; font-style: italic;">// The two conditions create one rule</span>
Rule bossDetectionRule <span style="color: #339933;">=</span> <span style="color: #000000; font-weight: bold;">new</span> BooleanRule<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
bossDetectionRule.<span style="color: #006633;">addCondition</span><span style="color: #009900;">&#40;</span>inOffice<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
bossDetectionRule.<span style="color: #006633;">addCondition</span><span style="color: #009900;">&#40;</span>bossPhone<span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #666666; font-style: italic;">// React when the rule matches</span>
bossDetectionRule.<span style="color: #006633;">addRuleListener</span><span style="color: #009900;">&#40;</span><span style="color: #000000; font-weight: bold;">new</span> RuleListener<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
   <span style="color: #000000; font-weight: bold;">public</span> <span style="color: #000066; font-weight: bold;">void</span> ruleChanged<span style="color: #009900;">&#40;</span>Rule rule, <span style="color: #000066; font-weight: bold;">boolean</span> matches<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
     <span style="color: #000000; font-weight: bold;">if</span> <span style="color: #009900;">&#40;</span>matches<span style="color: #009900;">&#41;</span> <span style="color: #009900;">&#123;</span>
        playWarningSound<span style="color: #009900;">&#40;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
        displayWarningMessage<span style="color: #009900;">&#40;</span><span style="color: #0000ff;">&quot;Warning: boss has arrived!&quot;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
     <span style="color: #009900;">&#125;</span>
<span style="color: #009900;">&#125;</span><span style="color: #009900;">&#125;</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span>
&nbsp;
<span style="color: #666666; font-style: italic;">// Activate rule</span>
bossDetectionRule.<span style="color: #006633;">setActive</span><span style="color: #009900;">&#40;</span><span style="color: #000066; font-weight: bold;">true</span><span style="color: #009900;">&#41;</span><span style="color: #339933;">;</span></pre></td></tr></table></div>

<p>In lines 2-3, the specific conditions are created. Since both conditions represent very generic cases (Bluetooth/WLAN device in range), ACAPI provides predefined conditions for them. In cases where more complex tests are desired, conditions can be extended via simple Java inheritance. Lines 6-8 combine the two conditions to a single Boolean rule, i.e. a rule that becomes either <em>true</em> or <em>false</em> depending on the status of its conditions. Since the application is supposed to react on changes in this particular rule, it registers itself as a listener in lines 11-17. When the rule is activated (line 20), it tells its conditions to register themselves at the corresponding sensors, which in turn get activated (if not already running). After this initialization, ACAPI notifies all registered listeners whenever the rule changes.</p>
<p>Depending on the status of the rules and conditions, i.e. on the device context, the application can change its appearance, behavior or internal state. In this case, it only plays a warning sound and displays a warning message (lines 14-15).</p>
<h4 id="Proof-of-Concept-Application" >4.2. Proof-of-Concept Application</h4>
<p>In order to test the implemented sensors and the rules engine of ACAPI, we developed a proof-of-concept application that implements a more sophisticated context driven use case.</p>
<p>A field service automation application that reacts upon the context the worker is in at the moment. This can be nicely done with the ACAPI and has a value for businesses. However, as this part of our project is not open source, I will not go into more detail here. </p>
<h3 id="Future-Work-and-Conclusion" >5. Future Work and Conclusion</h3>
<p>The Aware Context API provides a framework for building context-aware applications for mobile devices based on Java ME. By providing uniform interfaces to different sensors, the library allows the development of context-driven applications.</p>
<p>The idea and structure of ACAPI are very solid, however, the actual implementation is in a very early development stage. While most sensors and Boolean conditions/rules are already working on the test devices, the Fuzzy conditions and rules are yet to be implemented. The native part only covers Symbian S60 so far and lacks of stability. Hence, the future work will include the implementation of missing parts, testing as well as the documentation. </p>
<h3 id="Download-and-License" >A. Download and License</h3>
<p>ACAPI will be released as open source, possibly under GPL or a Creative Commons license. Since we have not finished cleaning up the code and commenting everything, the code is not available for download as yet. </p>
<p>However, since it will be open source anyway, I will give out the code upon request.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2010/08/22/aware-context-api-acapi-using-the-sensors-in-java-me/feed/</wfw:commentRss>
		<slash:comments>4</slash:comments>
		</item>
		<item>
		<title>Hybrid Clouds: A Comparison of Cloud Toolkits</title>
		<link>http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/</link>
		<comments>http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#comments</comments>
		<pubDate>Sat, 08 May 2010 20:32:11 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Distributed Systems]]></category>
		<category><![CDATA[Virtualization]]></category>
		<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Eucalyptus]]></category>
		<category><![CDATA[Hyper-V]]></category>
		<category><![CDATA[OpenNebula]]></category>
		<category><![CDATA[oVirt]]></category>
		<category><![CDATA[RHEV]]></category>
		<category><![CDATA[VMware]]></category>
		<category><![CDATA[XenServer]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=307</guid>
		<description><![CDATA[In the last few years, the importance of the Internet has risen constantly and made it indispensable for businesses and most individuals to be on-line around the clock. One of the greatest drivers of this development was and still is the shift of the traditional one-to-many Web to an advanced, participatory version of the Word [...]]]></description>
			<content:encoded><![CDATA[<p>In the last few years, the importance of the Internet has risen constantly and made it indispensable for businesses and most individuals to be on-line around the clock. One of the greatest drivers of this development was and still is the shift of the traditional one-to-many Web to an advanced, participatory version of the Word Wide Web. Rather than only making editorial information accessible to many users, the Web 2.0 encourages participation and enables user generated contributions. Leveraging this new paradigm, services like Flickr, Facebook, or Twitter have become very prominent examples for this development.</p>
<p>An essential part of this evolution, but mostly hidden to the end-consumer, is the set of tools that enable these large scale applications. <b>Cloud computing</b> is a relatively new technology that serves as underlying architecture for most of these platforms. By providing <a href="http://en.wikipedia.org/wiki/Virtualization">virtualized</a> computing resources as a service in a pay-as-you-go manner, cloud computing enables new business models and cost effective resource usage. Instead of having to maintain their own data center, companies can concentrate on their core business and purchase resources when needed. Especially when combining a privately maintained virtual infrastructure with publicly accessible clouds in a <b>hybrid cloud</b>, the technology can open up new opportunities for businesses and help consolidating resources.<br />
However, since cloud computing is a very new term, there are as many definitions of its components as there are opinions about its usefulness. Most of the corresponding technologies are only a few years old and the toolkits lack of maturity and interoperability.</p>
<p>This article introduces the basic concepts of cloud computing and discusses the technical requirements for setting up a hybrid cloud. It briefly looks into security concerns and outlines the status quo of current cloud technologies. In particular, it evaluates several existing cloud toolkits regarding its requirements, occurring problems and interoperability.</p>
<p><span id="more-307"></span></p>
<hr />
<b>Contents</b></p>
<div class="toc">
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#Cloud-Computing">1. Cloud Computing</a></p>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#Status-Quo">1.1. Status Quo</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#Definitions-and-Key-Characteristics">1.2. Definitions and Key Characteristics</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#Classifications">1.3. Classifications</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#toc-1-3-1-service-models-abstraction-classes">1.3.1. Service Models: Abstraction Classes</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/#toc-1-3-2-deployment-models-exposure-classes">1.3.2. Deployment Models: Exposure Classes</a></li>
</ol>
</li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#Hybrid-Clouds">2. Hybrid Clouds</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#Opportunities">2.1. Opportunities</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#Challenges-and-Issues">2.2. Challenges and Issues</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#toc-2-2-1-cost">2.2.1. Cost</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#toc-2-2-2-security-and-data-confidentiality">2.2.2. Security and Data Confidentiality</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#toc-2-2-3-availability">2.2.3. Availability</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/2/#Interoperability">2.2.4. Interoperability</a></li>
</ol>
</li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Cloud-Toolkits">3. Cloud Toolkits</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Market-Overview">3.1. Market Overview</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Technical-Requirements-and-Restrictions">3.2. Technical Requirements and Restrictions</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Hardware-Requirements">3.2.1. Hardware Requirements</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Operating-Systems-and-Software-Restrictions">3.2.2. Operating Systems and Software Restrictions</a></li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#OpenNebula-and-Eucalyptus">3.3. OpenNebula and Eucalyptus</a>
<ol>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#Eucalyptus">3.3.1. Eucalyptus</a></li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/3/#OpenNebula">3.3.2. OpenNebula</a></li>
</ol>
</li>
</ol>
</li>
<li><a href="http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/4/#Conclusion">4. Conclusion</a></li>
</ol>
</div>
<hr />
<b>Download as PDF</b>: This article is a slightly shortened version of my seminar paper. Feel free to download the <a href="http://www.philippheckel.com/files/hybrid-cloud-paper.pdf">original PDF version</a>, or the <a href="http://www.philippheckel.com/files/hybrid-cloud-presentation.pdf">presentation slides</a>.</p>
<hr />
<h3 id="Cloud-Computing" >1. Cloud Computing</h3>
<h4 id="Status-Quo" >1.1. Status Quo</h4>
<p>As newest concept in the development of distributed computing, cloud computing is often believed to be &#8220;<em>the next step in the evolution of the Internet</em>&#8221; (<a href="http://www.opencloudmanifesto.org/">Open Cloud Manifesto</a>). As foundation and enabler for <a href="http://en.wikipedia.org/wiki/Software_as_a_service">Software as a Service</a>, it delivers computing resources over the Internet and provides elastic scalability for any kind of application. While cluster and grid computing already allowed multiple computers to work together on complex tasks in a distributed manner, the cloud concept extends this idea even further: instead of regarding individual machines, cloud computing treats resources as a <em>utility</em>. That is computing time and storage are provisioned on-demand and paid per usage without the need for any upfront commitment.</p>
<p>As one of the first commercial providers of cloud services, Amazon launched a beta version of its <a href="http://aws.amazon.com/ec2/">Elastic Computing Cloud (EC2)</a> in August 2006 and announced production stability <a href="http://aws.typepad.com/aws/2008/10/big-day-for-ec2.html">in October 2008</a>. Google followed with a public beta of <a href="http://code.google.com/appengine/">App Engine</a> in <a href="http://googleappengine.blogspot.com/2008/04/introducing-google-app-engine-our-new.html">April 2008</a>, and Microsoft made its cloud platform <a href="http://www.microsoft.com/windowsazure/">Windows Azure</a> publicly available in <a href="http://blogs.msdn.com/windowsazure/archive/2010/02/01/windows-azure-platform-now-generally-available-in-21-countries.aspx">February 2010</a>. The well known alternatives to the commercial solutions are several open source cloud toolkits. Prominent examples include <a href="http://www.opennebula.org/">OpenNebula</a>, a project started by researchers of the University of Chicago and Madrid in 2008, as well as the <a href="http://www.eucalyptus.com/">Eucalyptus</a> cloud software, initiated by the University of California, Santa Barbara in 2007. </p>
<p>Even though there are already many commercial and open source cloud solutions, all of them are fairly young and have yet to prove their acceptance and durability. According to the <a href="http://www.opencloudmanifesto.org/">Open Cloud Manifesto</a>, the technology &#8220;<em>is still in its early stages, with much to learn and more experimentation to come</em>&#8220;. This particularly includes challenges that yet need to be overcome, e.g. data security within the cloud, or interoperability between different clouds. </p>
<h4 id="Definitions-and-Key-Characteristics" >1.2. Definitions and Key Characteristics</h4>
<p>Due to the actuality of the topic, there are several opinions about what cloud computing and its corresponding terms comprises. Some experts see the technology as &#8220;<em>one of the foundations of next generation computing</em>&#8221; (<a href="http://youtube.com/?v=6PNuQHUiV3Q">Tim O&#8217;Reilly, CEO of O&#8217;Reilly Media, 2008</a>), others believe that the term is just a buzz word to define &#8220;<em>everything that we currently do</em>&#8221; (<a href="http://youtube.com/?v=0FacYAI6DY0" title="What the Hell is Cloud Computing?">Larry Ellison, CEO of Oracle Corp., 2008</a>).</p>
<p>However, while the term is being criticized, there are still many intersecting definitions describing the technology. Some are broader than others and include not only the technical part, but also the services enabled by the cloud, i.e. SaaS applications. For the Laboratory of Distributed Systems of the University of California, Berkeley, for instance, &#8220;<em>cloud computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the data centers that provide those services</em>&#8221; (<a href="http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf">Armbrust et al., 2009</a>). Not contrary to that definition, but limiting the term to hardware and software, IBM describes cloud computing as &#8220;<em>an emerging computing paradigm where data and services reside in massively scalable data centers and can be ubiquitously accessed from any connected devices over the Internet</em>&#8221; (<a href="http://www.ibm.com/developerworks/xml/library/x-cloudpt1/">O&#8217;Neill, 2009</a>). </p>
<p>In this article, cloud computing is defined by the three key characteristics shared by many experts:</p>
<ul>
<li><b>Resource abstraction</b>: resources inside the cloud are not directly observable by the cloud user, but are virtualized using technologies like <a href="http://www.xen.org/">Xen</a> or <a href="http://www.linux-kvm.org/">KVM</a>, and can be accessed via an application programmable interface (API).</li>
<li><b>Elastic capacity</b>: the cloud appears to users as a pool of infinite capacity, being able to allocate and free resources on-demand. Scaling up and down avoids over- and under-utilization and thereby allows an optimal load.</li>
<li><b>Utility-based pricing</b>: storage, CPU time and network bandwidth are charged by the hour using a pay-per-use pricing model. Resources can be allocated almost instantaneously without any upfront commitment.</li>
</ul>
<h4 id="Classifications" >1.3. Classifications</h4>
<p>Similar to the attempts to define the term cloud computing, the categorization of it is rather difficult, if not &#8220;<em>impossible in the currently rapid evolution of the cloud landscape</em>&#8221; (<a href="http://portal.acm.org/citation.cfm?id=1564625">Lenk et al., 2009</a>). However, many papers classify cloud systems by their level of abstraction and their exposure to the Internet. </p>
<h5 id="toc-1-3-1-service-models-abstraction-classes">1.3.1. Service Models: Abstraction Classes</h5>
<p>In order create the illusion of infinite resources and elasticity, virtualization technology is needed. Depending on how abstracted resources are, different <em>service models</em> are differentiated (cmp. <a href="http://csrc.nist.gov/groups/SNS/cloud-computing/">NIST, 2009</a>, and <a href="http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.pdf">Armbrust et al., 2009</a>):</p>
<ul>
<li><b>Software as a Service (SaaS)</b>: at the highest level of abstraction, users are mostly unaware of the fact that are using cloud-enabled applications, and are hence not able to control the underlying resources. Instead, they simply use client interfaces such as web browsers. A popular example is the <a href="http://www.salesforce.com/">salesforce.com</a> CRM system.</li>
<li><b>Platform as a Service (PaaS)</b>: users are able to develop and deploy applications within the provider&#8217;s hosting environment, e.g. a Java application framework. Low-level resources are not controlled by the cloud user. Prominent example is the <a href="http://appengine.google.com/">Google App Engine</a>.
<li><b>Infrastructure as a Service (IaaS)</b>: at the lowest level of abstraction, cloud users have access to virtualized resources such as processing time, networking or storage. They are provided virtual machines and can run arbitrary software. Famous example is <a href="http://aws.amazon.com/ec2/">Amazon EC2</a>.
</ul>
<h5 id="toc-1-3-2-deployment-models-exposure-classes">1.3.2. Deployment Models: Exposure Classes</h5>
<p>Not every cloud is available for public use: depending on the level of exposure to the Internet, the following <em>deployment models</em> are differentiated (<a href="http://csrc.nist.gov/groups/SNS/cloud-computing/">NIST, 2009</a>):</p>
<ul>
<li><b>Public Cloud</b>: the cloud infrastructure is publicly accessible via an API. It is hosted by a cloud provider who sells its capacity using a pay-per-use payment model. All of the above mentioned examples are public clouds.
</li>
<li><b>Private Cloud</b>: the cloud infrastructure is hosted within the data center of an organization and used by local users only. It focuses on providing a flexible virtualized infrastructure rather than on selling the cloud resources.
</li>
<li><b>Hybrid Cloud</b>: the hybrid cloud approach extends the private cloud model by using both local and remote resources. It is usually used to handle flash crowds by scaling out when the local capacity is exhausted. This so called cloudbursting enables highly elastic environments. The key difference between private and hybrid clouds is &#8220;<em>the extension of service provider-oriented low cost cloud storage to the enterprise</em>&#8221; (<a href="http://cloudstoragestrategy.com/2010/01/cloud-storage-for-the-enterprise---part-2-the-hybrid-cloud.html">Lesem, 2010</a>). That is remote cloud resources are seamlessly integrated in the private cloud, and thereby create a hybrid cloud.</li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2010/05/08/hybrid-clouds-comparing-cloud-toolkits/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>WP-UN: WordPress version update notification with cron</title>
		<link>http://blog.philippheckel.com/2010/01/29/wp-un-wordpress-version-update-notification-with-cron/</link>
		<comments>http://blog.philippheckel.com/2010/01/29/wp-un-wordpress-version-update-notification-with-cron/#comments</comments>
		<pubDate>Fri, 29 Jan 2010 13:32:30 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Administration]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[Cron]]></category>
		<category><![CDATA[Mail]]></category>
		<category><![CDATA[Postfix]]></category>
		<category><![CDATA[Scripts]]></category>
		<category><![CDATA[WordPress]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=164</guid>
		<description><![CDATA[WordPress is a very popular open-source blog software and is used widely throughout the Internet. However, with great success comes great attack potential: like any other wide spread open-source software, WordPress is target for frequent hacking attacks and spam-bots. All the more important is it to always update the distribution to the latest release. As [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://wordpress.org/">WordPress</a> is a very popular open-source blog software and is used widely throughout the Internet. However, with great success comes great attack potential: like any other wide spread open-source software, WordPress is target for frequent hacking attacks and spam-bots. All the more important is it to always update the distribution to the latest release.</p>
<p>As Debian/Ubuntu user, I am spoiled when it comes to update management: <em>apt-get</em> updates most of my software, and <em>apticron</em> notifies me when updates are available. For WordPress however, the packaged versions of Debian/Ubuntu are really old and less adjustable which unfortunately makes a manual installation inevitable. While there are several automated WordPress update mechanisms out there, I couldn&#8217;t find a simple notify-on-update tool.</p>
<p>This post introduces the <em>WordPress Update Notifier</em> (WP-UN), a simple script that frequently compares the installed WordPress version with the latest available one. If a new version is available, it sends an e-mail to a given address.</p>
<p><span id="more-164"></span></p>
<h3 id="toc-update">Update</h3>
<p><strong>February &#8217;11</strong>: I updated the script so that it now uses the WordPress API. If you want, you can still download the <a href="http://blog.philippheckel.com/uploads/2010/01/wp-un-0.1">old version of WP-UN</a>, but since wordpress.org changed their download mechanisms, it does not work any more. </p>
<h3 id="toc-requirements">Requirements</h3>
<p>WP-UN is compatible with <strong>WordPress 2.5-3.x</strong>. It needs a  <strong>local mail server</strong> such as <a href="http://www.sendmail.org/">Sendmail</a> or <a href="http://www.postfix.org/">Postfix</a> to deliver the notification e-mail. </p>
<h3 id="toc-download-installation">Download &amp; Installation</h3>
<p>Download the script, save it to your preferred location and make it executable:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">wget</span> <span style="color: #660033;">-O</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>local<span style="color: #000000; font-weight: bold;">/</span>bin<span style="color: #000000; font-weight: bold;">/</span>wp-un \
          http:<span style="color: #000000; font-weight: bold;">//</span>blog.philippheckel.com<span style="color: #000000; font-weight: bold;">/</span>uploads<span style="color: #000000; font-weight: bold;">/</span><span style="color: #000000;">2010</span><span style="color: #000000; font-weight: bold;">/</span>01<span style="color: #000000; font-weight: bold;">/</span>wp-un
$ <span style="color: #c20cb9; font-weight: bold;">chmod</span> +x <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>local<span style="color: #000000; font-weight: bold;">/</span>bin<span style="color: #000000; font-weight: bold;">/</span>wp-un</pre></div></div>

<p>That&#8217;s it for the installation. The script can now be called by simply running <em>wp-un</em>.</p>
<p><strong>Download</strong>: <a href="http://blog.philippheckel.com/uploads/2010/01/wp-un">WP-UN 0.2, February 2011</a><br />
<strong>Old version</strong>: <a href="http://blog.philippheckel.com/uploads/2010/01/wp-un-0.1">WP-UN 0.1, January 2010</a> (broken!)</p>
<h3 id="toc-usage">Usage</h3>
<p>Now you can call the script with the following arguments:</p>
<ul>
<li><strong>&#8211;test</strong>: to test if the notification works, use the <em>&#8211;test</em> parameter (optional).</li>
<li><strong>INSTALL-DIR</strong>: the path to your local WordPress installation, for example /var/www/myblog.</li>
<li><strong>NOTIFY-EMAIL</strong>: the e-mail address of the person to notify if a new WordPress version is available.</li>
</ul>
<p>By default, the script is completely silent so that adding a cronjob doesn&#8217;t require output redirections. If, however, the <em>&#8211;test</em> option is given, it is more verbose and sends the notification e-mail in any case.</p>
<p>If a new WordPress version is available, the output looks something like this:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ wp-un --test /var/www/myblog admin@example.com
Checking installed version... WordPress 2.5.1
Checking latest version... WordPress 2.9.1
Update required; Sending notification to admin@example.com... done.</pre></div></div>

<p>If WordPress is up-to-date, WP-UN would normally not send any notification. If, however, the <em>&#8211;test</em> option is enabled, it sends the e-mail no matter what. In this case, the output will look like this:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ wp-un --test /var/www/myblog admin@example.com
Checking installed version... WordPress 2.9.1
Checking latest version... WordPress 2.9.1
Update not necessary; WordPress is up-to-date.
TEST-flag enabled: sending notfication to admin@example.com... done.</pre></div></div>

<p>The notification you receive will look like this:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;"> The WordPress installation on host example.com needs an update:
&nbsp;
   Installed Version: WordPress 2.5.1
                  at: /var/www/myblog
&nbsp;
      Latest Version: WordPress 2.9.1
            Download: http://www.wordpress.org/latest.tar.gz</pre></div></div>

<h3 id="toc-as-cronjob">As cronjob</h3>
<p>If you want to be notified as soon as a new version comes out, installing a cronjob is a good idea. Simply run <em>crontab -e</em> and add the following line to the file:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;"><span style="color: #000000;">0</span> <span style="color: #000000;">6</span> <span style="color: #000000; font-weight: bold;">*</span> <span style="color: #000000; font-weight: bold;">*</span> <span style="color: #000000; font-weight: bold;">*</span> <span style="color: #000000; font-weight: bold;">/</span>usr<span style="color: #000000; font-weight: bold;">/</span>local<span style="color: #000000; font-weight: bold;">/</span>bin<span style="color: #000000; font-weight: bold;">/</span>wp-un <span style="color: #000000; font-weight: bold;">/</span>var<span style="color: #000000; font-weight: bold;">/</span>www<span style="color: #000000; font-weight: bold;">/</span>myblog admin<span style="color: #000000; font-weight: bold;">@</span>example.com</pre></div></div>

<p>WP-UN will now run every morning at 6am and notify you if a new WordPress version is out there!</p>
<h3 id="toc-conclusion">Conclusion</h3>
<p>WP-UN is just one of many solutions and it&#8217;s only the work of one afternoon. However, it doesn&#8217;t need any additional software and keeps it simple. It serves its purpose and keeps my WordPress installation always up-to-date. If you have any suggestions or questions, feel free to comment below.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2010/01/29/wp-un-wordpress-version-update-notification-with-cron/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How to: Postfix as mail relay with greylisting support</title>
		<link>http://blog.philippheckel.com/2010/01/28/how-to-postfix-as-mail-relay-with-greylisting-support/</link>
		<comments>http://blog.philippheckel.com/2010/01/28/how-to-postfix-as-mail-relay-with-greylisting-support/#comments</comments>
		<pubDate>Thu, 28 Jan 2010 20:35:11 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Administration]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[Debian]]></category>
		<category><![CDATA[Greylisting]]></category>
		<category><![CDATA[Mail]]></category>
		<category><![CDATA[Postfix]]></category>
		<category><![CDATA[SQLgrey]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/?p=30</guid>
		<description><![CDATA[Greylisting is a very efficient technique for fighting spam and can reduce the spam messages in your mailbox by more than 90%. It uses the fact that most spammers only try delivering their spam-mails once, whereas real mail transfer agents (such as the ones regular e-mail service providers are using) try delivering each message up [...]]]></description>
			<content:encoded><![CDATA[<p><a href="http://www.greylisting.org/">Greylisting</a> is a very efficient technique for fighting spam and can reduce the spam messages in your mailbox by more than 90%. It uses the fact that most spammers only try delivering their spam-mails once, whereas real mail transfer agents (such as the ones regular e-mail service providers are using) try delivering each message up to 4-5 days before they give up. </p>
<p>I have always wondered why most ESPs don&#8217;t offer greylisting for their mailboxes, but only rely on less effective and resource-hungry post-retrieval filter methods. Unfortunately, my e-mail provider is one of them so that I get at least a couple of spam mails a day &#8230;</p>
<p>Luckily, it is very easy to set up your own mail relay with greylisting support, i.e. a mail server that simply forwards the mail to your real provider once it passes the greylist-filter. </p>
<p>This little tutorial describes how to set up <a href="http://www.postfix.org">Postfix</a> and <a href="http://sqlgrey.sourceforge.net/">SQLgrey</a> as mail relay.</p>
<p><span id="more-30"></span></p>
<h3 id="toc-1-what-you-need">1. What you need</h3>
<ul>
<li>A dedicated or virtual private server with SSH root access.</li>
<li>Access to the DNS entries of your domain for adjusting the MX record; in this post called <em>example.com</em></li>
</ul>
<h3 id="toc-2-how-it-works">2. How it works</h3>
<p>If you have outsourced all e-mail services to a service provider like I have, the MX record of your domain usually points to your provider&#8217;s mail server. That is, your mails go directly to the mail server of your provider, e.g. <strong>Google&#8217;s mail server → your provider&#8217;s mail server</strong>. That is, your DNS configuration looks something like this:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ dig example.com mx
...
example.com.		IN	MX	50 mx0.example.com.
mx0.example.com.	IN	A	(your provider's mail server IP)
...</pre></div></div>

<p>In order to pre-process mails with greylisting and blacklisting, your server will handle mails as intermediary, i.e., mails will always traverse your server first; in the above case something like <strong>Google&#8217;s mail server → your mail server → your provider&#8217;s mail server</strong>. Consequently, the MX record has to be changed to the IP address of your server:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ dig example.com mx
...
example.com.		IN	MX	50 mx0.example.com.
mx0.example.com.	IN	A	(your server IP)
...</pre></div></div>

<p><strong>But</strong>, first things first: we need to configure our server before we change the DNS records!</p>
<h3 id="toc-3-installation-configuration">3. Installation &amp; Configuration</h3>
<p>If you have a Debian based system, install Postfix, SQLgrey and MySQL using apt-get:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> postfix sqlgrey mysql-server</pre></div></div>

<p>This will install:</p>
<ul>
<li><strong><a href="http://www.postfix.org/">Postfix</a></strong>: a fully functioning MTA which will be configured as mail relay, i.e., instead of storing arriving mails on the system, it will just greylist them and then forward them to their real destination (your provider&#8217;s mail server).</li>
<li><strong><a href="http://sqlgrey.sourceforge.net/">SQLgrey</a></strong>: a SQL-based greylisting add-on for Postfix. Before accepting mails blindly, Postfix will ask the SQLgrey daemon whether to accept the mail or not. SQLgrey keeps track of mail delivery attempts and only replies with success if the foreign MTA tried delivering the mail at least twice.</li>
<li><strong><a href="http://www.mysql.com/">MySQL</a></strong>: a RDBMS which will be used as back-end for storing Postfix&#8217;s routing tables and SQLgrey&#8217;s caching tables. Both Postfix and SQLgrey also support other back-ends such as <a href="http://www.postgresql.org/">PostgreSQL</a>.</li>
</ul>
<h4 id="toc-3-1-configuring-sqlgrey">3.1. Configuring SQLgrey</h4>
<p>SQLgrey&#8217;s config files reside in <em>/etc/sqlgrey/</em>, the main configuration happens in <em>/etc/sqlgrey/sqlgrey.conf</em>. The file is well documented and offers many possibilities. </p>
<p>The most important options are:</p>
<ul>
<li><strong>inet</strong>: IP address and port to bind the daemon to, default is 127.0.0.1:2501</li>
<li><strong>db_*</strong>: database connection details, i.e., database, user and password</li>
<li><strong>greymethod</strong>: defines which <a href="http://en.wikipedia.org/wiki/Classful_network#Introduction_of_address_classes">IP class</a> to use for greylisting. Especially important for big e-mail service providers since the same mail might be delivered from two different IP addresses (Class C greylisting recommended!).</li>
<li><strong>optmethod</strong>: defines if greylisting is enabled by default (optout), or has to be enabled specifically for each address or domain (optin).</li>
</ul>
<h5 id="toc-3-1-1-config-file-etcsqlgreysqlgrey-conf">3.1.1. Config file /etc/sqlgrey/sqlgrey.conf</h5>
<p>My configuration looks like this:</p>

<div class="wp_syntax"><div class="code"><pre class="properties" style="font-family:monospace;"><span style="color: #000080; font-weight:bold;">inet</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 10101 </span><span style="color: #808080; font-style: italic;"># bind to localhost:10101</span>
<span style="color: #000080; font-weight:bold;">reconnect_delay</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 5 </span><span style="color: #808080; font-style: italic;"># no reconnect before 5 minutes</span>
<span style="color: #000080; font-weight:bold;">max_connect_age</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 24 </span><span style="color: #808080; font-style: italic;"># no reconnect after 24 hours</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># database settings</span>
<span style="color: #000080; font-weight:bold;">db_type</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> mysql</span>
<span style="color: #000080; font-weight:bold;">db_name</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> sqlgrey</span>
<span style="color: #000080; font-weight:bold;">db_host</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> localhost</span>
<span style="color: #000080; font-weight:bold;">db_user</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> sqlgrey</span>
<span style="color: #000080; font-weight:bold;">db_pass</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> sqlgreypassword</span>
<span style="color: #000080; font-weight:bold;">db_cleandelay</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 1800 </span>
<span style="color: #000080; font-weight:bold;">clean_method</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> sync </span><span style="color: #808080; font-style: italic;"># 'async' is said to be buggy</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># greylist by class C network. eg: 2.3.4.6 connection</span>
<span style="color: #808080; font-style: italic;"># accepted if 2.3.4.145 did connect earlier</span>
<span style="color: #000080; font-weight:bold;">greymethod</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> classc</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># one must optin to have its (incoming) messages being greylisted</span>
<span style="color: #000080; font-weight:bold;">optmethod</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> optin</span></pre></div></div>

<h5 id="toc-3-1-2-database">3.1.2. Database</h5>
<p>SQLgrey has a fixed database structure which is set up automatically when the script is started. All that needs to be done is to create a new database <em>sqlgrey</em> with a corresponding user. You can do this manually, or with a tool like <a href="http://www.phpmyadmin.net/">phpMyAdmin</a>:</p>

<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">USER</span> <span style="color: #ff0000;">'sqlgrey'</span>@<span style="color: #ff0000;">'localhost'</span> <span style="color: #993333; font-weight: bold;">IDENTIFIED</span> <span style="color: #993333; font-weight: bold;">BY</span> <span style="color: #ff0000;">'sqlgreypassword'</span>;
<span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">DATABASE</span> <span style="color: #993333; font-weight: bold;">IF</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">EXISTS</span> <span style="color: #ff0000;">`sqlgrey`</span> ;
<span style="color: #993333; font-weight: bold;">GRANT</span> <span style="color: #993333; font-weight: bold;">ALL</span> PRIVILEGES <span style="color: #993333; font-weight: bold;">ON</span> <span style="color: #ff0000;">`sqlgrey`</span> <span style="color: #66cc66;">.</span> <span style="color: #66cc66;">*</span> <span style="color: #993333; font-weight: bold;">TO</span> <span style="color: #ff0000;">'sqlgrey'</span>@<span style="color: #ff0000;">'localhost'</span>;
<span style="color: #993333; font-weight: bold;">FLUSH</span> PRIVILEGES;</pre></div></div>

<h5 id="toc-3-1-3-populating-the-database">3.1.3. Populating the database</h5>
<p>SQLgrey automatically creates the required tables when it starts for the first time. So start the daemon using the provided init.d-script:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #000000; font-weight: bold;">/</span>etc<span style="color: #000000; font-weight: bold;">/</span>init.d<span style="color: #000000; font-weight: bold;">/</span>sqlgrey start</pre></div></div>

<p>This creates a couple of tables in the <em>sqlgrey</em>-database. For our purpose and configuration, the tables <em>optin_email</em> and <em>optin_domain</em> are most interesting because only domains and e-mail addresses in these tables will be greylisted.</p>
<p>For our example, we will enable greylisting for the whole domain <em>example.com</em>:</p>

<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">INSERT</span> <span style="color: #993333; font-weight: bold;">INTO</span> <span style="color: #ff0000;">`sqlgrey`</span><span style="color: #66cc66;">.</span><span style="color: #ff0000;">`optin_domain`</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`domain`</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">VALUES</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'example.com'</span><span style="color: #66cc66;">&#41;</span>;</pre></div></div>

<p>That&#8217;s it for SQLgrey. Once we connect it to Postfix, it&#8217;ll provide us with the greylisting service we want. </p>
<h4 id="toc-3-2-configuring-postfix">3.2. Configuring Postfix</h4>
<p>Postfix is a very flexible and powerful mail transfer agent (MTA) and can be used as final destination, or mail forwarder (mail relay). For this scenario, Postfix will be a <em>mail relay</em> which only forwards an e-mail if</p>
<ul>
<li>its recipient is listed in the database</li>
<li>and it passes the greylisting-filter</li>
</ul>
<p>The configuration files of Postfix reside in <em>/etc/postfix/</em>. The most interesting file for our purpose is <em>/etc/postfix/main.cf</em>.</p>
<p>In order to not be confused by all the more or less useful config parameters, the file shown here is <em>minimal</em>, i.e., you <em>cannot</em> remove any parameter without major consequences. Details to each of them can be found in the <a href="http://www.postfix.org/postconf.5.html">Postfix configuration man-page</a>.</p>
<p>The most important parameters for the configuration as mail relay are:</p>
<ul>
<li><strong><a href="http://www.postfix.org/postconf.5.html#myhostname">myhostname</a></strong>: this defines your hostname, i.e., in this case <em>relay.example.com</em>. Regarding the hostname, two things must be considered very thoroughly:
<ol>
<li>The hostname must resolve to the IP address of your server, i.e., make sure you don&#8217;t state a fake host here. Postfix uses this value as <a href="http://en.wikipedia.org/wiki/Simple_Mail_Transfer_Protocol">HELO/EHLO</a> identification, and some mail servers might <strong>reject</strong> your mails if  this value doesn&#8217;t resolve to your server&#8217;s IP address.</li>
<li>If you use your top level domain here, e.g. example.com, some mail servers might additionally perform a <a href="http://en.wikipedia.org/wiki/MX_record">MX lookup</a> and match your server&#8217;s IP address with the one of the MX record. In case the MX record points to a different IP address than the A record of the TLD, foreign servers might also <strong>reject</strong> all your mails. In my case, this resulted in log entries like this:

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">postfix/smtp: to=&lt;john.doe@example.com&gt;, 
  relay=mx.my-esp.tld:25,  status=bounced 
  (host mx.somedomain.com[1.2.3.4] refused to talk to me: 
  550 Forged HELO: you are not example.com)</pre></div></div>

<p>As you can see, the foreign hosts suspected me of forging the HELO name, and denied relaying my mails.</li>
</ol>
</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#relay_domains">relay_domains</a></strong>: this option links to the database table <em>relay_domains</em> and defines the domains managed by this mail server. If a recipient-domain is not in this table, Postfix will reject the mail. For this example, the domain <em>example.com</em> must be added to this DB table.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#relay_recipient_maps">relay_recipient_maps</a></strong>: this option links to the database table <em>relay_recipients</em> and defines the e-mail addresses managed by this mail server. If a recipient address is not in this table, Postfix will reject the mail. This option is closely linked to <em>relay_domains</em> and will not work without it!
<p>In this case, only one address will be added to this table: <em>john.doe@example.com</em>.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#transport_maps">transport_maps</a></strong>: this option links to the database table <em>transport</em> and defines to which mail server incoming mails will be forwarded. Routing can happen address- or domain-based.
<p>In this case, mails for <em>example.com</em> shall be forwarded to our provider&#8217;s mail server, i.e., this table must have an entry of the form <em>example.com</em> → <em>smtp:[mx.my-esp.tld]</em>.</p>
<p>For details on the value format, read the <a href="http://www.postfix.org/transport.5.html">transport man page</a>.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#smtpd_recipient_restrictions">smtpd_recipient_restrictions</a></strong>: this is where the actual magic happens. This option checks the RCPT TO field of each incoming mail, i.e., the recipient, and then queries the greylisting service. Its options closely relate to the <em>relay_*</em>-tables from above:
<ul>
<li><strong><a href="http://www.postfix.org/postconf.5.html#permit_mynetworks">permit_mynetworks</a></strong> allows local applications to send e-mails. If you have web sites running on localhost that may use e-mail, do not remove this option.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#reject_unauth_destination">reject_unauth_destination</a></strong> queries the <em>relay_domains</em> SQL table, i.e., it checks whether a the incoming recipient domain is relayed by our server.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#reject_unlisted_recipient">reject_unlisted_recipient</a></strong> queries the <em>relay_recipients</em> SQL table to find out if the exact address is relayed.</li>
<li><strong><a href="http://www.postfix.org/postconf.5.html#check_policy_service">check_policy_service</a></strong> queries the SQLgrey daemon which in turn either allows or rejects the mail.</li>
</ul>
</li>
</ul>
<h5 id="toc-3-2-1-config-file-etcpostfixmain-cf">3.2.1 Config file /etc/postfix/main.cf</h5>
<p>Here&#8217;s the minimal <em>main.cf</em> config file to make Postfix a mail relay with greylisting support:</p>

<div class="wp_syntax"><div class="code"><pre class="properties" style="font-family:monospace;"><span style="color: #808080; font-style: italic;"># This is a minimal main.cf config file. Make sure to read the above </span>
<span style="color: #808080; font-style: italic;"># comments so you understand what each option means.</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># server name; must resolve to your server's IP address</span>
<span style="color: #000080; font-weight:bold;">myhostname</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> relay.example.com</span>
&nbsp;
<span style="color: #808080; font-style: italic;"># avoid warning message 'dict_nis_init: NIS ...'</span>
<span style="color: #000080; font-weight:bold;">alias_maps</span> <span style="color: #000000;">=</span> 
&nbsp;
<span style="color: #000080; font-weight:bold;">relay_domains</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> mysql:/etc/postfix/mysql_relay_domains.cf</span>
<span style="color: #000080; font-weight:bold;">relay_recipient_maps</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> mysql:/etc/postfix/mysql_relay_recipient_maps.cf</span>
<span style="color: #000080; font-weight:bold;">transport_maps</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> mysql:/etc/postfix/mysql_transport_maps.cf</span>
&nbsp;
<span style="color: #000080; font-weight:bold;">smtpd_recipient_restrictions</span> <span style="color: #000000;">=</span>
        permit_mynetworks,
        reject_unauth_destination,
        reject_unlisted_recipient,
        check_policy_service inet:127.0.0.1:<span style="">10101</span></pre></div></div>

<h5 id="toc-3-2-2-database">3.2.2. Database</h5>
<p>Postfix is very flexible when it comes to address and route handling. In fact, its configuration doesn&#8217;t need a database back-end at all. However, using a SQL database makes everything much easier. I decided to use a very straight forward database structure which directly relates to Postfix&#8217; configuration options. </p>
<p>First, create a database <em>postfix</em> and a corresponding read-only user:</p>

<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">USER</span> <span style="color: #ff0000;">'postfix'</span>@<span style="color: #ff0000;">'127.0.0.1'</span> <span style="color: #993333; font-weight: bold;">IDENTIFIED</span> <span style="color: #993333; font-weight: bold;">BY</span> <span style="color: #ff0000;">'postfixpassword'</span>;
<span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">DATABASE</span> <span style="color: #993333; font-weight: bold;">IF</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">EXISTS</span> <span style="color: #ff0000;">`postfix`</span> ;
<span style="color: #993333; font-weight: bold;">GRANT</span> <span style="color: #993333; font-weight: bold;">SELECT</span> <span style="color: #993333; font-weight: bold;">ON</span> <span style="color: #ff0000;">`postfix`</span> <span style="color: #66cc66;">.</span> <span style="color: #66cc66;">*</span> <span style="color: #993333; font-weight: bold;">TO</span> <span style="color: #ff0000;">'postfix'</span>@<span style="color: #ff0000;">'127.0.0.1'</span>;
<span style="color: #993333; font-weight: bold;">FLUSH</span> PRIVILEGES;</pre></div></div>

<p><strong>Note</strong>: It is important that you use <em>127.0.0.1</em> as host, and not <em>localhost</em>, because Postfix runs in a <a href="http://en.wikipedia.org/wiki/Chroot">chroot</a>-environment and wouldn&#8217;t be able to access <em>localhost</em>. </p>
<p>After setting up the database, add the following three tables:</p>

<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">TABLE</span> <span style="color: #ff0000;">`relay_domains`</span> <span style="color: #66cc66;">&#40;</span>
  <span style="color: #ff0000;">`domain`</span> <span style="color: #993333; font-weight: bold;">VARCHAR</span><span style="color: #66cc66;">&#40;</span><span style="color: #cc66cc;">255</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span><span style="color: #66cc66;">,</span>
  <span style="color: #ff0000;">`active`</span> enum<span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span><span style="color: #ff0000;">'n'</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span> <span style="color: #993333; font-weight: bold;">DEFAULT</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span>
  <span style="color: #993333; font-weight: bold;">PRIMARY</span> <span style="color: #993333; font-weight: bold;">KEY</span>  <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`domain`</span><span style="color: #66cc66;">&#41;</span>
<span style="color: #66cc66;">&#41;</span>;
&nbsp;
<span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">TABLE</span> <span style="color: #ff0000;">`relay_recipients`</span> <span style="color: #66cc66;">&#40;</span>
  <span style="color: #ff0000;">`email`</span> <span style="color: #993333; font-weight: bold;">VARCHAR</span><span style="color: #66cc66;">&#40;</span><span style="color: #cc66cc;">255</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span><span style="color: #66cc66;">,</span>
  <span style="color: #ff0000;">`active`</span> enum<span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span><span style="color: #ff0000;">'n'</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span> <span style="color: #993333; font-weight: bold;">DEFAULT</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span>
  <span style="color: #993333; font-weight: bold;">PRIMARY</span> <span style="color: #993333; font-weight: bold;">KEY</span>  <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`email`</span><span style="color: #66cc66;">&#41;</span>
<span style="color: #66cc66;">&#41;</span>;
&nbsp;
<span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">TABLE</span> <span style="color: #ff0000;">`transport`</span> <span style="color: #66cc66;">&#40;</span>
  <span style="color: #ff0000;">`pattern`</span> <span style="color: #993333; font-weight: bold;">VARCHAR</span><span style="color: #66cc66;">&#40;</span><span style="color: #cc66cc;">255</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span><span style="color: #66cc66;">,</span>
  <span style="color: #ff0000;">`relay`</span> <span style="color: #993333; font-weight: bold;">VARCHAR</span><span style="color: #66cc66;">&#40;</span><span style="color: #cc66cc;">255</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span><span style="color: #66cc66;">,</span>
  <span style="color: #ff0000;">`active`</span> enum<span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span><span style="color: #ff0000;">'n'</span><span style="color: #66cc66;">&#41;</span> <span style="color: #993333; font-weight: bold;">NOT</span> <span style="color: #993333; font-weight: bold;">NULL</span> <span style="color: #993333; font-weight: bold;">DEFAULT</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">,</span>
  <span style="color: #993333; font-weight: bold;">PRIMARY</span> <span style="color: #993333; font-weight: bold;">KEY</span>  <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`pattern`</span><span style="color: #66cc66;">&#41;</span>
<span style="color: #66cc66;">&#41;</span>;</pre></div></div>

<p>In order to connect Postfix with the database, we need to create the three config files specified above: /etc/postfix/mysql_*.cf:</p>

<div class="wp_syntax"><div class="code"><pre class="properties" style="font-family:monospace;"><span style="color: #808080; font-style: italic;"># /etc/postfix/mysql_relay_domains.cf</span>
<span style="color: #000080; font-weight:bold;">hosts</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 127.0.0.1</span>
<span style="color: #000080; font-weight:bold;">user</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix-read</span>
<span style="color: #000080; font-weight:bold;">password</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfixpassword</span>
<span style="color: #000080; font-weight:bold;">dbname</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix</span>
<span style="color: #000080; font-weight:bold;">query</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> SELECT domain FROM relay_domains WHERE domain='%s' AND active='y'</span></pre></div></div>


<div class="wp_syntax"><div class="code"><pre class="properties" style="font-family:monospace;"><span style="color: #808080; font-style: italic;"># /etc/postfix/mysql_relay_recipient_maps.cf</span>
<span style="color: #000080; font-weight:bold;">hosts</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 127.0.0.1</span>
<span style="color: #000080; font-weight:bold;">user</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix-read</span>
<span style="color: #000080; font-weight:bold;">password</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfixpassword</span>
<span style="color: #000080; font-weight:bold;">dbname</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix</span>
<span style="color: #000080; font-weight:bold;">query</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> SELECT email FROM relay_recipients WHERE email='%s' AND active='y'</span></pre></div></div>


<div class="wp_syntax"><div class="code"><pre class="properties" style="font-family:monospace;"><span style="color: #808080; font-style: italic;"># /etc/postfix/mysql_transport_maps.cf</span>
<span style="color: #000080; font-weight:bold;">hosts</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> 127.0.0.1</span>
<span style="color: #000080; font-weight:bold;">user</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix-read</span>
<span style="color: #000080; font-weight:bold;">password</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfixpassword</span>
<span style="color: #000080; font-weight:bold;">dbname</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> postfix</span>
<span style="color: #000080; font-weight:bold;">query</span> <span style="color: #000000;">=</span><span style="color: #008000; font-weight:bold;"> SELECT relay FROM transport WHERE pattern='%s' AND active='y'</span></pre></div></div>

<p>Before we can now start testing our server, we need to compile these config files to Postfix compatible lookup tables. Do that by running the following command:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ postmap <span style="color: #000000; font-weight: bold;">/</span>etc<span style="color: #000000; font-weight: bold;">/</span>postfix<span style="color: #000000; font-weight: bold;">/</span>mysql_<span style="color: #000000; font-weight: bold;">*</span>.cf</pre></div></div>

<h5 id="toc-3-2-3-populate-the-database">3.2.3. Populate the database</h5>
<p>Now we have to fill Postfix&#8217; database with the domains and addresses we&#8217;d like to relay. In particular, that means we have to add <em>example.com</em> to <em>relay_domains</em> and <em>transport</em>, and the full addresses to <em>relay_recipients</em>:</p>

<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">INSERT</span> <span style="color: #993333; font-weight: bold;">INTO</span> <span style="color: #ff0000;">`postfix`</span><span style="color: #66cc66;">.</span><span style="color: #ff0000;">`relay_domains`</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`domain`</span> <span style="color: #66cc66;">,</span><span style="color: #ff0000;">`active`</span><span style="color: #66cc66;">&#41;</span>
   <span style="color: #993333; font-weight: bold;">VALUES</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'example.com'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">&#41;</span>;
&nbsp;
<span style="color: #993333; font-weight: bold;">INSERT</span> <span style="color: #993333; font-weight: bold;">INTO</span> <span style="color: #ff0000;">`postfix`</span><span style="color: #66cc66;">.</span><span style="color: #ff0000;">`relay_recipients`</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`email`</span> <span style="color: #66cc66;">,</span><span style="color: #ff0000;">`active`</span><span style="color: #66cc66;">&#41;</span>
   <span style="color: #993333; font-weight: bold;">VALUES</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'john.doe@example.com'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">&#41;</span>;
&nbsp;
<span style="color: #993333; font-weight: bold;">INSERT</span> <span style="color: #993333; font-weight: bold;">INTO</span> <span style="color: #ff0000;">`postfix`</span><span style="color: #66cc66;">.</span><span style="color: #ff0000;">`transport`</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">`pattern`</span> <span style="color: #66cc66;">,</span><span style="color: #ff0000;">`relay`</span> <span style="color: #66cc66;">,</span><span style="color: #ff0000;">`active`</span><span style="color: #66cc66;">&#41;</span>
   <span style="color: #993333; font-weight: bold;">VALUES</span> <span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'example.com'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'smtp:[mx.my-e-mail-service-provider.tld]'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'y'</span><span style="color: #66cc66;">&#41;</span>;</pre></div></div>

<p>The entry structure for each table is different. Please refer to the Postfix manual for details (cp. <a href="http://www.postfix.org/transport.5.html">transport</a>, <a href="http://www.postfix.org/postconf.5.html#relay_domains">relay_domains</a>, and <a href="http://www.postfix.org/postconf.5.html#relay_recipient_maps">relay_recipient_maps</a>).</p>
<p><strong>Note</strong>: the database structure above is not optimal since it requires redundant entries in three different tables. Even though the structure is not perfect, I have chosen this layout to make it easily understandable!</p>
<h3 id="toc-4-test-your-server">4. Test your server</h3>
<p>After this short configuration period it&#8217;s now time to finally start the Postfix server:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #000000; font-weight: bold;">/</span>etc<span style="color: #000000; font-weight: bold;">/</span>init.d<span style="color: #000000; font-weight: bold;">/</span>postfix start</pre></div></div>

<p>To make sure you didn&#8217;t make any mistakes in the configuration, you should now check the log files. Postfix and SQLgrey both use syslog so that you should be able to determine the system&#8217;s status like this:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">tail</span> <span style="color: #660033;">-n</span> <span style="color: #000000;">20</span> <span style="color: #660033;">-f</span> <span style="color: #000000; font-weight: bold;">/</span>var<span style="color: #000000; font-weight: bold;">/</span>log<span style="color: #000000; font-weight: bold;">/</span>syslog</pre></div></div>

<p>If the log doesn&#8217;t show any errors, we can now try if everything works as expected. To do so, simply connect to the server from your home computer via telnet:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ telnet relay.example.com 25
Connected to relay.example.com.
Escape character is '^]'.
220 relay.example.com ESMTP Postfix
HELO somebody
250 relay.example.com
MAIL FROM: some@address.tld
250 2.1.0 Ok
RCPT TO: john.doe@example.com
450 4.7.1 &lt;john.doe@example.com&gt;: 
   Recipient address rejected: Greylisted for 5 minutes</pre></div></div>

<p>If Postfix replies with a 450 error code, i.e., relay temporarily denied, everything works just fine. On the server side, the log should output something like this:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
&nbsp;
sqlgrey: grey: new: 4.3.2(4.3.2.1), some@address.tld-&gt;john.doe@example.com 
postfix/smtpd: NOQUEUE: reject: RCPT from 1-2-3-4.your-isp.tld[4.3.2.1]: 
    450 4.7.1 &lt;john.doe@example.com&gt;: 
    Recipient address rejected: Greylisted for 5 minutes; 
    from=&lt;some@address.tld&gt; to=&lt;john.doe@example.com&gt; ....
&nbsp;
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]</pre></div></div>

<p>Wait 5 minutes and try connecting again via telnet. This time, SQLgrey will detect that this is your second delivery attempt and add the sender e-mail and its IP address to the automatic white list (AWL). Postfix will accept your mail and forward it to your provider&#8217;s mail server (according to the <em>transport</em>-table):</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
&nbsp;
sqlgrey: grey: reconnect ok: 4.3.2(4.3.2.1), 
    some@address.tld -&gt; john.doe@example.com (00:22:42) 
sqlgrey: grey: from awl: 4.3.2, some@address.tld added 
&nbsp;
postfix/smtpd: client=1-2-3-4.your-isp.tld[4.3.2.1]
postfix/cleanup: message-id=&lt;201001...@relay.example.com&gt;
postfix/qmgr: from=&lt;some@address.tld&gt;, size=422, ...
postfix/smtp: to=&lt;john.doe@example.com&gt;, 
    relay=mx.my-esp.tld[12.34.56.78]:25, status=sent, ...
postfix/qmgr: removed
&nbsp;
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]</pre></div></div>

<h3 id="toc-5-go-live-change-the-dns-record">5. Go live: change the DNS record</h3>
<p>Play around a little and make sure that everything works as expected. If it does, change the DNS record like described above, i.e., set the MX record of the domains to be relayed to your server&#8217;s IP address.</p>
<p><strong>Note</strong>: For the first few mails, you should definitely watch the logs. If anything goes wrong, you can always change back the MX record. But be aware that DNS changes might take up to 48h!</p>
<p>If you have any questions, please comment below. I am open for suggestions!</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2010/01/28/how-to-postfix-as-mail-relay-with-greylisting-support/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Dell Latitude E6400: the Ubuntu fan and noise problem</title>
		<link>http://blog.philippheckel.com/2009/09/21/dell-latitude-e6400-the-ubuntu-fan-and-noise-problem/</link>
		<comments>http://blog.philippheckel.com/2009/09/21/dell-latitude-e6400-the-ubuntu-fan-and-noise-problem/#comments</comments>
		<pubDate>Mon, 21 Sep 2009 13:16:32 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Dell Latitude]]></category>
		<category><![CDATA[Ubuntu]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/2009/09/21/dell-latitude-e6400-the-ubuntu-fan-and-noise-problem/</guid>
		<description><![CDATA[A couple of days ago, my 4 year old laptop (HP nx8220) decided that it was time to retire and refused to switch on a couple of times. Even though it works most of the time, I can&#8217;t rely on it anymore and will therefore look for new notebooks in the next week. On my [...]]]></description>
			<content:encoded><![CDATA[<p>A couple of days ago, my 4 year old laptop (HP nx8220) decided that it was time to retire and refused to switch on a couple of times. Even though it works most of the time, I can&#8217;t rely on it anymore and will therefore look for new notebooks in the next week.</p>
<p>On my journey through the Web I looked into the details of various business notebooks. I stumbled across the <strong>Dell Latitude E6400</strong>, which looks nice and has everything I wanted. Unfortunately, it also seems to have problems with <strong>Ubuntu</strong>. In particular, the fan seems to run all the time (and not too slow, but very loud) &#8211; as many forum posts and user reviews prove [<a href="http://www.linlap.com/wiki/dell+latitude+e6400">1</a>,<a href="http://karussell.wordpress.com/2009/07/03/dell-latitude-e6400-the-perfect-linux-notebook-or-is-ubuntu-the-perfect-linux/">2</a>,<a href="http://en.community.dell.com/forums/t/19282841.aspx">3</a>,<a href="http://en.community.dell.com/forums/t/19249116.aspx">4</a>].</p>
<p>Since I couldn&#8217;t find a solution, I decided to ask the Dell support via their support chat.<br />
<span id="more-25"></span></p>
<p><strong>Update Nov/09</strong>: It appears that a BIOS update addresses the heat problem. Please refer to the <a href="http://en.community.dell.com/forums/t/19247293.aspx?PageIndex=12">community forum post</a> (thanks to <a href="#comment-12920">chato</a>) for details. </p>
<p>Here&#8217;s what I found out. Note: This is a chat with the German Dell support from September 21, 2009, translated into English.</p>
<blockquote><p>
<strong>Me</strong>:<br />
The Latitude E6400 has (like many forum posts say) a big problem with Ubuntu. In particular, the fan is always on and makes a lot of noise.</p>
<p><strong>Agent</strong>:<br />
Welcome to the Dell Chat.</p>
<p><strong>Me</strong>:<br />
Hallo. I&#8217;d like to know if there already is a solution for this problem. I was thinking about buying a E6400, but I couldn&#8217;t find any solutions for the fan/noise problem in Ubuntu.</p>
<p><strong>Agent</strong>:<br />
Since we do not support Ubuntu on this device, Dell does not work on a solution. Ubuntu is currently only supported on netbooks. There is no solution.</p>
<p><strong>Me</strong>:<br />
Okay. Do you think there will be a solution in the future? Is Dell planning to support Ubuntu on the E6400? Or never?!</p>
<p><strong>Agent</strong>:<br />
As I said, we don&#8217;t sell the device with Ubuntu.</p>
<p><strong>Me</strong>:<br />
Okay. Thank you for the honest answer. I&#8217;m gonna have to buy a Thinkpad then &#8230; Have a nice day.</p>
<p><strong>Agent</strong>:<br />
I&#8217;m sorry. But we don&#8217;t support Linux.</p>
<p><strong>Me</strong>:<br />
It&#8217;s not your fault. Thanks anyway. Good bye.</p>
<p><strong>Agent</strong>:<br />
I&#8217;m so sorry.
</p></blockquote>
<p>So the outcome is: <strong>The Dell Latitude E6400 doesn&#8217;t support Ubuntu and is NOT planning to solve the fan/noise problem.</strong>. That means, I&#8217;m gonna have to buy one of those ugly Thinkpads. But what other solutions are there?! None!</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2009/09/21/dell-latitude-e6400-the-ubuntu-fan-and-noise-problem/feed/</wfw:commentRss>
		<slash:comments>8</slash:comments>
		</item>
		<item>
		<title>Extract text from PDF files</title>
		<link>http://blog.philippheckel.com/2009/08/09/extract-text-from-pdf-files/</link>
		<comments>http://blog.philippheckel.com/2009/08/09/extract-text-from-pdf-files/#comments</comments>
		<pubDate>Sun, 09 Aug 2009 17:17:05 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Office]]></category>
		<category><![CDATA[PDF]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/2009/08/09/extract-text-from-pdf-files/</guid>
		<description><![CDATA[Adobe&#8217;s Portable Document Format (PDF) has reached great popularity over the last years and is the number one format for easy document exchange. It comes with great features such as embeddable images and multimedia, but also has rather unpleasant properties. The so called Security Features represent a simple Digital Rights Management (DRM) system and allow [...]]]></description>
			<content:encoded><![CDATA[<p>Adobe&#8217;s Portable Document Format (PDF) has reached great popularity over the last years and is the number one format for easy document exchange. It comes with great features such as embeddable images and multimedia, but also has rather unpleasant properties. The so called <em>Security Features</em> represent a simple Digital Rights Management (DRM) system and allow PDF authors to restrict the file usage. Using the DRM system, authors can allow or deny actions such as printing a file, commenting or copying content.</p>
<p>Even though this is a good idea for some situations, most of the times, it&#8217;s just annoying: Collecting ideas for seminar papers or a thesis, for instance, is almost impossible without being able to Copy &amp; Paste certain paragraphs from the PDF. </p>
<p><span id="more-24"></span></p>
<p>Fortunately, Linux can solve this problem with a simple tool called <strong>pdf to text</strong>. This command line tool simply strips all text from the PDF file and saves it to a given text-file.</p>
<h3 id="toc-installation">Installation</h3>
<p>The tool is part of the package <strong>poppler-utils</strong> and can be installed via your favorite package manager, e.g. apt-get:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">apt-get</span> <span style="color: #c20cb9; font-weight: bold;">install</span> poppler-utils</pre></div></div>

<h3 id="toc-extract-text-from-pdf-files">Extract text from PDF files</h3>
<p>This is also pretty simple and the man-page gives the instructions: <em>pdftotext [options] &lt;PDF&gt; [&lt;text-file&gt;]</em>.</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ pdftotext PDF-file-with-copy-and-paste-restriction.pdf</pre></div></div>

<p>In case you&#8217;d like to perform this for every PDF-file in a folder (recursive search), simple do that:</p>

<div class="wp_syntax"><div class="code"><pre class="bash" style="font-family:monospace;">$ <span style="color: #c20cb9; font-weight: bold;">find</span> <span style="color: #660033;">-name</span> <span style="color: #ff0000;">'*.pdf'</span> <span style="color: #660033;">-exec</span> pdftotext <span style="color: #ff0000;">&quot;{}&quot;</span> \;</pre></div></div>

<p>After executing the command, there will be a *.txt-file for each PDF file in the folder, &#8211; containing the plain-text of the corresponding PDF file.</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2009/08/09/extract-text-from-pdf-files/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Simsafe: Simple command-line password safe</title>
		<link>http://blog.philippheckel.com/2009/04/07/simsafe-simple-command-line-password-safe/</link>
		<comments>http://blog.philippheckel.com/2009/04/07/simsafe-simple-command-line-password-safe/#comments</comments>
		<pubDate>Tue, 07 Apr 2009 00:20:29 +0000</pubDate>
		<dc:creator>Philipp C. Heckel</dc:creator>
				<category><![CDATA[Linux]]></category>
		<category><![CDATA[Security]]></category>
		<category><![CDATA[GPG]]></category>

		<guid isPermaLink="false">http://blog.philippheckel.com/2009/04/07/simsafe-simple-command-line-password-safe/</guid>
		<description><![CDATA[Nowadays, it appears to me as if almost everything in the big and fancy world of IT comes with the need to sign up and create an account. Every little online tool, every social networking site and of course every instant messenger account. System administrators hits it even harder: The setup of a server machine [...]]]></description>
			<content:encoded><![CDATA[<p>Nowadays, it appears to me as if almost everything in the big and fancy world of IT comes with the need to sign up and create an account. Every little online tool, every social networking site and of course every instant messenger account. System administrators hits it even harder: The setup of a server machine requires to create lots of different users for every kind of service, &#8212; be it Postfix, Sendmail, Courier, MySQL, PostgreSQL, etc. Most of them require some kind of super-user password or account.</p>
<p>This is where a password manager comes in handy: Open the password vault by typing in the master password, put in all you secrets and crucial information, save it and be happy. <em>As if!</em></p>
<p>Almost every password manager I found on the Web was crowded out by details so that it took minutes to add a single account. What I wanted was something like a text-file with password &#8212; and that&#8217;s what I made: <strong>A simple command-line password safe</strong>.</p>
<p><span id="more-22"></span></p>
<h3 id="toc-how-it-works">How it works</h3>
<p>Simsafe is nothing else but a simple Perl script wrapped around the symmetric encryption functions of GPG. Every box with GPG and Perl installed can hence use the script (= every linux box!). And even if the <em>simsafe</em>-script is not installed on the system, a simple <em>gpg &#8211;decrypt FILE</em> shows the plain text contents. That is, Simsafe files are nothing more but <strong>text-files with password!</strong></p>
<h3 id="toc-usage">Usage</h3>
<p>It&#8217;s simple and really easy to use.<br />
Syntax:</p>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ simsafe FILE</pre></div></div>

<h4 id="toc-create-a-new-password-safe-file">Create a new password safe file</h4>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ simsafe mySafe
simsafe: Creating a new password safe 'mySafe'
simsafe: Please enter the new password: (type in your password)
simsafe: Please confirm the password: (confirm your password)
simsafe: Executing editor vi ...
  // VI or your favourite editor opens
  // and you can edit the plain text file
simsafe: Encrypting with GPG ...</pre></div></div>

<h4 id="toc-addingremoving-entries-to-the-password-vault">Adding/Removing entries to the password vault</h4>

<div class="wp_syntax"><div class="code"><pre class="text" style="font-family:monospace;">$ simsafe mySafe
simsafe: Please enter the safe password: (type in your password)
simsafe: Decrypting with GPG ...
simsafe: Executing editor vi ...
  // VI or your favourite editor opens
  // and you can edit the plain text file
simsafe: Safe unchanged.</pre></div></div>

<h3 id="toc-get-it">Get it!</h3>
<p>It&#8217;s a reeeaallly small script and this post is possible longer than the perl-file itself, but here it is. Simply download it, make it executable and put it somewhere in your PATH.</p>
<p><strong>Download:</strong> <a href="/uploads/2009/04/simsafe">Simsafe v0.1, Apr. 2009</a></p>
]]></content:encoded>
			<wfw:commentRss>http://blog.philippheckel.com/2009/04/07/simsafe-simple-command-line-password-safe/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>

