For developers, context-awareness can be both a blessing and a curse. While the mobile operating systems iPhone OS and Android come with relatively good sensor-support, the vast majority of devices deal with Java ME’s basic and heterogeneous sensor functionalities.

The Aware Context API (ACAPI) aims to bridge this gap by providing a framework for building context aware applications for mobile devices based on Java ME. In this article, I’d like to introduce ACAPI, its structure and usage briefly. Please feel free to comment.

1. Motivation and Goals

ACAPI is designed to allow easy and homogeneous access to the different sensors of the mobile device. It creates an abstraction between the available sensors and their usage so that developers can focus on the business logic rather than on how to use the sensor implementations.

Example: A mobile application shall notify the user if another (previously defined) person comes into his or her range, e.g. if the boss arrives at the office.

Using the standard Java ME interfaces, developers have to get to know the different APIs and write a lot of code to solve this or similar problems. In this use case, the application needs to determine its position (the office), monitor the available devices around it (the phone of the boss) and be able to notify the user when both events occur.

The Aware Context API aims to solve this reoccurring problem with an easy-to-use event-based framework that allows defining rules for available sensor data. Using ACAPI, the given example can be solved easily by defining a rule and an action that is triggered when the rule matches:

• Rule: Wi-Fi “OfficeWLAN” available AND Bluetooth device “BossPhone” available
• Action: Notify user, e.g. by playing a sound

2. Development Team and Scope

The ACAPI project was developed within the scope of a team project as part of the curriculum of a Master of Science Degree at the University of Mannheim. The project was a team-effort, carried out over a 1-year duration at the Chair of Business Administration and Information Systems, under the supervision of Prof. Armin Heinzl and his research assistants Erik Hemmer and Sebastian Stuckenberg.

The project team consisted of Lars Bakker, Philipp Heckel (myself), Obie Modisane, Benjamin Schubert and Moritz Wächter.

3. Aware Context API (ACAPI)

The Aware Context API is well-structured and is very easy to understand. It is easily extendible and supports a broad range of devices. It is mainly based on Java ME, but has native parts whenever needed (e.g. for Wi-Fi, battery or telephony).

3.1. ACAPI Structure

ACAPI is horizontally structured into 3 different layers:

• Sensor: On the lowest level, the sensors gather data about the current status and context of the phone. A Wi-Fi sensor, for instance, collects available devices and it issues an event whenever the data changes. Applications can either hook themselves directly into the sensor events or use higher abstractions (conditions and rules).
• Condition: In order to evaluate a single sensor, conditions compare the sensor’s properties to given values. They can become either true or false. A location condition, for example, becomes true if the phone gets into the range of certain coordinates. Similar to a sensor, a condition issues an event when the value changes (from true to false, or vice versa).
• Rule: To express more than one condition, rules can combine conditions to a more complex logical expression. In the above example, the rule only matches if both conditions match (“in the office” and “boss phone available”).

ACAPI Structure (simplified and incomplete!): The Aware Context API is layer-based. Each of the components is easily extendible and has event listeners to react on changes. This chart shows the interdependence of the different layers.

Using this layered structure, ACAPI fundamentally changes the development strategy of mobile applications. Instead of predefining a screen and/or process flow, applications are event-driven. Whenever a rule changes its state (match vs. no match), the application can react by displaying a different screen, notifying the user, or by performing other actions.

Besides the horizontal division, the API is also vertically divided in the two logical parts Boolean and Fuzzy. While the Boolean part assumes correct sensors, the Fuzzy conditions and rules take measurement errors and inaccuracy into account. While the Boolean conditions and rules can only become true or false, their Fuzzy counterparts implement a score-based system that only triggers when a certain threshold is reached. This is particularly relevant for sensors that supply accuracy data, e.g. GPS sensors.

3.2. Implemented Sensors

The current code base of ACAPI includes several predefined sensors, including the most common: Bluetooth and GPS. Most sensors are entirely based on Java ME and will work on any phone that supports the corresponding JSR. However, since Java ME does not provide access to some functionalities, a few native implementations are required (e.g. for Wi-Fi, battery status or telephony status).

The following sensors are already implemented:

• Battery Sensor (native S60): This sensor monitors the status of the battery (%) and the charger (enum value, e.g. on-battery, or plugged-in). There is currently only a Symbian S60 implementation for this sensor since Java ME does not allow access to the battery data.
• Bluetooth Sensor: This sensor monitors available devices, e.g. phones or laptops. It can react on joining or leaving devices.
• Custom Sensor: This sensor allows the integration of business logic in ACAPI, so that rules do not only include actual sensor data, but also virtual business sensor data.
• Location Sensor (GPS and Wi-Fi; partially native S60): This sensor monitors the position and the speed. It uses GPS and Wi-Fi triangulation to get a fast and accurate position. Since Java ME does not allow access to the wireless sensor, the Wi-Fi part is native S60 code.
• Noise Level Sensor: This sensor monitors the noise level (in decibels) of the surrounding area.
• Time Sensor: This sensor delivers the current time and can react on date and time changes.
• Wireless Sensor (native S60): This sensor monitors the available Wi-Fi devices, i.e. access points. It can react on joining and leaving devices. In combination with a web service, it can be used to estimate the position.

There are many other possible sensors that could be implemented using the available abstract classes. Examples include an orientation sensor (react on device movement) or a telephony sensor (react on calls, SMS etc.).

4. Example Usage

Having discussed the structure of the Aware Context API, the following section elaborates the above-mentioned example even further. It explains the scenario and shows specific example code.

4.1. Example Scenario and Code

As already briefly mentioned above, the example scenario for demonstrating the API is very simple: The application shall display a warning message and play a warning sound when the boss arrives at the office.

The two conditions depicted in the diagram above are combined in one Boolean rule, i.e. the rule only becomes true if both of the conditions match.

Similar to the API concept, its actual usage is also very simple. The following code snippet shows how to implement the above example in a regular Java ME application.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

// Conditions
Condition inOffice = new WirelessNearCondition("OfficeWLAN");
Condition bossPhone = new BluetoothNearCondition("BossPhone");
 
// The two conditions create one rule
Rule bossDetectionRule = new BooleanRule();
bossDetectionRule.addCondition(inOffice);
bossDetectionRule.addCondition(bossPhone);
 
// React when the rule matches
bossDetectionRule.addRuleListener(new RuleListener() {
   public void ruleChanged(Rule rule, boolean matches) {
     if (matches) {
        playWarningSound();
        displayWarningMessage("Warning: boss has arrived!");
     }
}});
 
// Activate rule
bossDetectionRule.setActive(true);

In lines 2-3, the specific conditions are created. Since both conditions represent very generic cases (Bluetooth/WLAN device in range), ACAPI provides predefined conditions for them. In cases where more complex tests are desired, conditions can be extended via simple Java inheritance. Lines 6-8 combine the two conditions to a single Boolean rule, i.e. a rule that becomes either true or false depending on the status of its conditions. Since the application is supposed to react on changes in this particular rule, it registers itself as a listener in lines 11-17. When the rule is activated (line 20), it tells its conditions to register themselves at the corresponding sensors, which in turn get activated (if not already running). After this initialization, ACAPI notifies all registered listeners whenever the rule changes.

Depending on the status of the rules and conditions, i.e. on the device context, the application can change its appearance, behavior or internal state. In this case, it only plays a warning sound and displays a warning message (lines 14-15).

4.2. Proof-of-Concept Application

In order to test the implemented sensors and the rules engine of ACAPI, we developed a proof-of-concept application that implements a more sophisticated context driven use case.

A field service automation application that reacts upon the context the worker is in at the moment. This can be nicely done with the ACAPI and has a value for businesses. However, as this part of our project is not open source, I will not go into more detail here.

5. Future Work and Conclusion

The Aware Context API provides a framework for building context-aware applications for mobile devices based on Java ME. By providing uniform interfaces to different sensors, the library allows the development of context-driven applications.

The idea and structure of ACAPI are very solid, however, the actual implementation is in a very early development stage. While most sensors and Boolean conditions/rules are already working on the test devices, the Fuzzy conditions and rules are yet to be implemented. The native part only covers Symbian S60 so far and lacks of stability. Hence, the future work will include the implementation of missing parts, testing as well as the documentation.

A. Download and License

ACAPI will be released as open source, possibly under GPL or a Creative Commons license. Since we have not finished cleaning up the code and commenting everything, the code is not available for download as yet.

However, since it will be open source anyway, I will give out the code upon request.

An essential part of this evolution, but mostly hidden to the end-consumer, is the set of tools that enable these large scale applications. Cloud computing is a relatively new technology that serves as underlying architecture for most of these platforms. By providing virtualized computing resources as a service in a pay-as-you-go manner, cloud computing enables new business models and cost effective resource usage. Instead of having to maintain their own data center, companies can concentrate on their core business and purchase resources when needed. Especially when combining a privately maintained virtual infrastructure with publicly accessible clouds in a hybrid cloud, the technology can open up new opportunities for businesses and help consolidating resources.
However, since cloud computing is a very new term, there are as many definitions of its components as there are opinions about its usefulness. Most of the corresponding technologies are only a few years old and the toolkits lack of maturity and interoperability.

This article introduces the basic concepts of cloud computing and discusses the technical requirements for setting up a hybrid cloud. It briefly looks into security concerns and outlines the status quo of current cloud technologies. In particular, it evaluates several existing cloud toolkits regarding its requirements, occurring problems and interoperability.

1. Cloud Computing

1.1. Status Quo

As newest concept in the development of distributed computing, cloud computing is often believed to be “the next step in the evolution of the Internet” (Open Cloud Manifesto). As foundation and enabler for Software as a Service, it delivers computing resources over the Internet and provides elastic scalability for any kind of application. While cluster and grid computing already allowed multiple computers to work together on complex tasks in a distributed manner, the cloud concept extends this idea even further: instead of regarding individual machines, cloud computing treats resources as a utility. That is computing time and storage are provisioned on-demand and paid per usage without the need for any upfront commitment.

As one of the first commercial providers of cloud services, Amazon launched a beta version of its Elastic Computing Cloud (EC2) in August 2006 and announced production stability in October 2008. Google followed with a public beta of App Engine in April 2008, and Microsoft made its cloud platform Windows Azure publicly available in February 2010. The well known alternatives to the commercial solutions are several open source cloud toolkits. Prominent examples include OpenNebula, a project started by researchers of the University of Chicago and Madrid in 2008, as well as the Eucalyptus cloud software, initiated by the University of California, Santa Barbara in 2007.

Even though there are already many commercial and open source cloud solutions, all of them are fairly young and have yet to prove their acceptance and durability. According to the Open Cloud Manifesto, the technology “is still in its early stages, with much to learn and more experimentation to come“. This particularly includes challenges that yet need to be overcome, e.g. data security within the cloud, or interoperability between different clouds.

1.2. Definitions and Key Characteristics

Due to the actuality of the topic, there are several opinions about what cloud computing and its corresponding terms comprises. Some experts see the technology as “one of the foundations of next generation computing” (Tim O’Reilly, CEO of O’Reilly Media, 2008), others believe that the term is just a buzz word to define “everything that we currently do” (Larry Ellison, CEO of Oracle Corp., 2008).

However, while the term is being criticized, there are still many intersecting definitions describing the technology. Some are broader than others and include not only the technical part, but also the services enabled by the cloud, i.e. SaaS applications. For the Laboratory of Distributed Systems of the University of California, Berkeley, for instance, “cloud computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the data centers that provide those services” (Armbrust et al., 2009). Not contrary to that definition, but limiting the term to hardware and software, IBM describes cloud computing as “an emerging computing paradigm where data and services reside in massively scalable data centers and can be ubiquitously accessed from any connected devices over the Internet” (O’Neill, 2009).

In this article, cloud computing is defined by the three key characteristics shared by many experts:

• Resource abstraction: resources inside the cloud are not directly observable by the cloud user, but are virtualized using technologies like Xen or KVM, and can be accessed via an application programmable interface (API).
• Elastic capacity: the cloud appears to users as a pool of infinite capacity, being able to allocate and free resources on-demand. Scaling up and down avoids over- and under-utilization and thereby allows an optimal load.
• Utility-based pricing: storage, CPU time and network bandwidth are charged by the hour using a pay-per-use pricing model. Resources can be allocated almost instantaneously without any upfront commitment.

1.3. Classifications

Similar to the attempts to define the term cloud computing, the categorization of it is rather difficult, if not “impossible in the currently rapid evolution of the cloud landscape” (Lenk et al., 2009). However, many papers classify cloud systems by their level of abstraction and their exposure to the Internet.

1.3.1. Service Models: Abstraction Classes

In order create the illusion of infinite resources and elasticity, virtualization technology is needed. Depending on how abstracted resources are, different service models are differentiated (cmp. NIST, 2009, and Armbrust et al., 2009):

• Software as a Service (SaaS): at the highest level of abstraction, users are mostly unaware of the fact that are using cloud-enabled applications, and are hence not able to control the underlying resources. Instead, they simply use client interfaces such as web browsers. A popular example is the salesforce.com CRM system.
• Platform as a Service (PaaS): users are able to develop and deploy applications within the provider’s hosting environment, e.g. a Java application framework. Low-level resources are not controlled by the cloud user. Prominent example is the Google App Engine.
• Infrastructure as a Service (IaaS): at the lowest level of abstraction, cloud users have access to virtualized resources such as processing time, networking or storage. They are provided virtual machines and can run arbitrary software. Famous example is Amazon EC2.

1.3.2. Deployment Models: Exposure Classes

Not every cloud is available for public use: depending on the level of exposure to the Internet, the following deployment models are differentiated (NIST, 2009):

• Public Cloud: the cloud infrastructure is publicly accessible via an API. It is hosted by a cloud provider who sells its capacity using a pay-per-use payment model. All of the above mentioned examples are public clouds.
• Private Cloud: the cloud infrastructure is hosted within the data center of an organization and used by local users only. It focuses on providing a flexible virtualized infrastructure rather than on selling the cloud resources.
• Hybrid Cloud: the hybrid cloud approach extends the private cloud model by using both local and remote resources. It is usually used to handle flash crowds by scaling out when the local capacity is exhausted. This so called cloudbursting enables highly elastic environments. The key difference between private and hybrid clouds is “the extension of service provider-oriented low cost cloud storage to the enterprise” (Lesem, 2010). That is remote cloud resources are seamlessly integrated in the private cloud, and thereby create a hybrid cloud.

As Debian/Ubuntu user, I am spoiled when it comes to update management: apt-get updates most of my software, and apticron notifies me when updates are available. For WordPress however, the packaged versions of Debian/Ubuntu are really old and less adjustable which unfortunately makes a manual installation inevitable. While there are several automated WordPress update mechanisms out there, I couldn’t find a simple notify-on-update tool.

This post introduces the WordPress Update Notifier (WP-UN), a simple script that frequently compares the installed WordPress version with the latest available one. If a new version is available, it sends an e-mail to a given address.

Requirements

WP-UN needs a local mail server such as Sendmail or Postfix to deliver the notification e-mail.

Download & Installation

Download the script, save it to your preferred location and make it executable:

$ wget -O /usr/local/bin/wp-un \
          http://blog.philippheckel.com/uploads/2010/01/wp-un
$ chmod +x /usr/local/bin/wp-un

That’s it for the installation. The script can now be called by simply running wp-un.

Download: WP-UN 0.1, January 2010

Usage

Now you can call the script with the following arguments:

• –test: to test if the notification works, use the –test parameter (optional).
• INSTALL-DIR: the path to your local WordPress installation, for example /var/www/myblog.
• NOTIFY-EMAIL: the e-mail address of the person to notify if a new WordPress version is available.

By default, the script is completely silent so that adding a cronjob doesn’t require output redirections. If, however, the –test option is given, it is more verbose and sends the notification e-mail in any case.

If a new WordPress version is available, the output looks something like this:

$ wp-un --test /var/www/myblog admin@example.com
Checking installed version... WordPress 2.5.1
Checking latest version... WordPress 2.9.1
Update required; Sending notification to admin@example.com... done.

If WordPress is up-to-date, WP-UN would normally not send any notification. If, however, the –test option is enabled, it sends the e-mail no matter what. In this case, the output will look like this:

$ wp-un --test /var/www/myblog admin@example.com
Checking installed version... WordPress 2.9.1
Checking latest version... WordPress 2.9.1
Update not necessary; WordPress is up-to-date.
TEST-flag enabled: sending notfication to admin@example.com... done.

The notification you receive will look like this:

 The WordPress installation on host example.com needs an update:
 
   Installed Version: WordPress 2.5.1
                  at: /var/www/myblog
 
      Latest Version: WordPress 2.9.1
            Download: http://www.wordpress.org/latest.tar.gz

As cronjob

If you want to be notified as soon as a new version comes out, installing a cronjob is a good idea. Simply run crontab -e and add the following line to the file:

0 6 * * * /usr/local/bin/wp-un /var/www/myblog admin@example.com

WP-UN will now run every morning at 6am and notify you if a new WordPress version is out there!

Conclusion

WP-UN is just one of many solutions and it’s only the work of one afternoon. However, it doesn’t need any additional software and keeps it simple. It serves its purpose and keeps my WordPress installation always up-to-date. If you have any suggestions or questions, feel free to comment below.>

I have always wondered why most ESPs don’t offer greylisting for their mailboxes, but only rely on less effective and resource-hungry post-retrieval filter methods. Unfortunately, my e-mail provider is one of them so that I get at least a couple of spam mails a day …

Luckily, it is very easy to set up your own mail relay with greylisting support, i.e. a mail server that simply forwards the mail to your real provider once it passes the greylist-filter.

This little tutorial describes how to set up Postfix and SQLgrey as mail relay.

1. What you need

• A dedicated or virtual private server with SSH root access.
• Access to the DNS entries of your domain for adjusting the MX record; in this post called example.com

2. How it works

If you have outsourced all e-mail services to a service provider like I have, the MX record of your domain usually points to your provider’s mail server. That is, your mails go directly to the mail server of your provider, e.g. Google’s mail server → your provider’s mail server. That is, your DNS configuration looks something like this:

$ dig example.com mx
...
example.com.		IN	MX	50 mx0.example.com.
mx0.example.com.	IN	A	(your provider's mail server IP)
...

In order to pre-process mails with greylisting and blacklisting, your server will handle mails as intermediary, i.e., mails will always traverse your server first; in the above case something like Google’s mail server → your mail server → your provider’s mail server. Consequently, the MX record has to be changed to the IP address of your server:

$ dig example.com mx
...
example.com.		IN	MX	50 mx0.example.com.
mx0.example.com.	IN	A	(your server IP)
...

But, first things first: we need to configure our server before we change the DNS records!

3. Installation & Configuration

If you have a Debian based system, install Postfix, SQLgrey and MySQL using apt-get:

$ apt-get install postfix sqlgrey mysql-server

This will install:

• Postfix: a fully functioning MTA which will be configured as mail relay, i.e., instead of storing arriving mails on the system, it will just greylist them and then forward them to their real destination (your provider’s mail server).
• SQLgrey: a SQL-based greylisting add-on for Postfix. Before accepting mails blindly, Postfix will ask the SQLgrey daemon whether to accept the mail or not. SQLgrey keeps track of mail delivery attempts and only replies with success if the foreign MTA tried delivering the mail at least twice.
• MySQL: a RDBMS which will be used as back-end for storing Postfix’s routing tables and SQLgrey’s caching tables. Both Postfix and SQLgrey also support other back-ends such as PostgreSQL.

3.1. Configuring SQLgrey

SQLgrey’s config files reside in /etc/sqlgrey/, the main configuration happens in /etc/sqlgrey/sqlgrey.conf. The file is well documented and offers many possibilities.

The most important options are:

• inet: IP address and port to bind the daemon to, default is 127.0.0.1:2501
• db_*: database connection details, i.e., database, user and password
• greymethod: defines which IP class to use for greylisting. Especially important for big e-mail service providers since the same mail might be delivered from two different IP addresses (Class C greylisting recommended!).
• optmethod: defines if greylisting is enabled by default (optout), or has to be enabled specifically for each address or domain (optin).

3.1.1. Config file /etc/sqlgrey/sqlgrey.conf

My configuration looks like this:

inet = 10101 # bind to localhost:10101
reconnect_delay = 5 # no reconnect before 5 minutes
max_connect_age = 24 # no reconnect after 24 hours
 
# database settings
db_type = mysql
db_name = sqlgrey
db_host = localhost
db_user = sqlgrey
db_pass = sqlgreypassword
db_cleandelay = 1800 
clean_method = sync # 'async' is said to be buggy
 
# greylist by class C network. eg: 2.3.4.6 connection
# accepted if 2.3.4.145 did connect earlier
greymethod = classc
 
# one must optin to have its (incoming) messages being greylisted
optmethod = optin

3.1.2. Database

SQLgrey has a fixed database structure which is set up automatically when the script is started. All that needs to be done is to create a new database sqlgrey with a corresponding user. You can do this manually, or with a tool like phpMyAdmin:

CREATE USER 'sqlgrey'@'localhost' IDENTIFIED BY 'sqlgreypassword';
CREATE DATABASE IF NOT EXISTS `sqlgrey` ;
GRANT ALL PRIVILEGES ON `sqlgrey` . * TO 'sqlgrey'@'localhost';
FLUSH PRIVILEGES;

3.1.3. Populating the database

SQLgrey automatically creates the required tables when it starts for the first time. So start the daemon using the provided init.d-script:

$ /etc/init.d/sqlgrey start

This creates a couple of tables in the sqlgrey-database. For our purpose and configuration, the tables optin_email and optin_domain are most interesting because only domains and e-mail addresses in these tables will be greylisted.

For our example, we will enable greylisting for the whole domain example.com:

INSERT INTO `sqlgrey`.`optin_domain` (`domain`) VALUES ('example.com');

That’s it for SQLgrey. Once we connect it to Postfix, it’ll provide us with the greylisting service we want.

3.2. Configuring Postfix

Postfix is a very flexible and powerful mail transfer agent (MTA) and can be used as final destination, or mail forwarder (mail relay). For this scenario, Postfix will be a mail relay which only forwards an e-mail if

• its recipient is listed in the database
• and it passes the greylisting-filter

The configuration files of Postfix reside in /etc/postfix/. The most interesting file for our purpose is /etc/postfix/main.cf.

In order to not be confused by all the more or less useful config parameters, the file shown here is minimal, i.e., you cannot remove any parameter without major consequences. Details to each of them can be found in the Postfix configuration man-page.

The most important parameters for the configuration as mail relay are:

• myhostname: this defines your hostname, i.e., in this case relay.example.com. Regarding the hostname, two things must be considered very thoroughly:
  1. The hostname must resolve to the IP address of your server, i.e., make sure you don’t state a fake host here. Postfix uses this value as HELO/EHLO identification, and some mail servers might reject your mails if this value doesn’t resolve to your server’s IP address.
  2. If you use your top level domain here, e.g. example.com, some mail servers might additionally perform a MX lookup and match your server’s IP address with the one of the MX record. In case the MX record points to a different IP address than the A record of the TLD, foreign servers might also reject all your mails. In my case, this resulted in log entries like this:

     postfix/smtp: to=<john.doe@example.com>, 
       relay=mx.my-esp.tld:25,  status=bounced 
       (host mx.somedomain.com[1.2.3.4] refused to talk to me: 
       550 Forged HELO: you are not example.com)

     As you can see, the foreign hosts suspected me of forging the HELO name, and denied relaying my mails.

• relay_domains: this option links to the database table relay_domains and defines the domains managed by this mail server. If a recipient-domain is not in this table, Postfix will reject the mail. For this example, the domain example.com must be added to this DB table.
• relay_recipient_maps: this option links to the database table relay_recipients and defines the e-mail addresses managed by this mail server. If a recipient address is not in this table, Postfix will reject the mail. This option is closely linked to relay_domains and will not work without it!

  In this case, only one address will be added to this table: john.doe@example.com.

• transport_maps: this option links to the database table transport and defines to which mail server incoming mails will be forwarded. Routing can happen address- or domain-based.

  In this case, mails for example.com shall be forwarded to our provider’s mail server, i.e., this table must have an entry of the form example.com → smtp:[mx.my-esp.tld].

  For details on the value format, read the transport man page.

• smtpd_recipient_restrictions: this is where the actual magic happens. This option checks the RCPT TO field of each incoming mail, i.e., the recipient, and then queries the greylisting service. Its options closely relate to the relay_*-tables from above:
  • permit_mynetworks allows local applications to send e-mails. If you have web sites running on localhost that may use e-mail, do not remove this option.
  • reject_unauth_destination queries the relay_domains SQL table, i.e., it checks whether a the incoming recipient domain is relayed by our server.
  • reject_unlisted_recipient queries the relay_recipients SQL table to find out if the exact address is relayed.
  • check_policy_service queries the SQLgrey daemon which in turn either allows or rejects the mail.

3.2.1 Config file /etc/postfix/main.cf

Here’s the minimal main.cf config file to make Postfix a mail relay with greylisting support:

# This is a minimal main.cf config file. Make sure to read the above 
# comments so you understand what each option means.
 
# server name; must resolve to your server's IP address
myhostname = relay.example.com
 
# avoid warning message 'dict_nis_init: NIS ...'
alias_maps = 
 
relay_domains = mysql:/etc/postfix/mysql_relay_domains.cf
relay_recipient_maps = mysql:/etc/postfix/mysql_relay_recipient_maps.cf
transport_maps = mysql:/etc/postfix/mysql_transport_maps.cf
 
smtpd_recipient_restrictions =
        permit_mynetworks,
        reject_unauth_destination,
        reject_unlisted_recipient,
        check_policy_service inet:127.0.0.1:10101

3.2.2. Database

Postfix is very flexible when it comes to address and route handling. In fact, its configuration doesn’t need a database back-end at all. However, using a SQL database makes everything much easier. I decided to use a very straight forward database structure which directly relates to Postfix’ configuration options.

First, create a database postfix and a corresponding read-only user:

CREATE USER 'postfix'@'127.0.0.1' IDENTIFIED BY 'postfixpassword';
CREATE DATABASE IF NOT EXISTS `postfix` ;
GRANT SELECT ON `postfix` . * TO 'postfix'@'127.0.0.1';
FLUSH PRIVILEGES;

Note: It is important that you use 127.0.0.1 as host, and not localhost, because Postfix runs in a chroot-environment and wouldn’t be able to access localhost.

After setting up the database, add the following three tables:

CREATE TABLE `relay_domains` (
  `domain` varchar(255) NOT NULL,
  `active` enum('y','n') NOT NULL DEFAULT 'y',
  PRIMARY KEY  (`domain`)
);
 
CREATE TABLE `relay_recipients` (
  `email` varchar(255) NOT NULL,
  `active` enum('y','n') NOT NULL DEFAULT 'y',
  PRIMARY KEY  (`email`)
);
 
CREATE TABLE `transport` (
  `pattern` varchar(255) NOT NULL,
  `relay` varchar(255) NOT NULL,
  `active` enum('y','n') NOT NULL DEFAULT 'y',
  PRIMARY KEY  (`pattern`)
);

In order to connect Postfix with the database, we need to create the three config files specified above: /etc/postfix/mysql_*.cf:

# /etc/postfix/mysql_relay_domains.cf
hosts = 127.0.0.1
user = postfix-read
password = postfixpassword
dbname = postfix
query = SELECT domain FROM relay_domains WHERE domain='%s' AND active='y'

# /etc/postfix/mysql_relay_recipient_maps.cf
hosts = 127.0.0.1
user = postfix-read
password = postfixpassword
dbname = postfix
query = SELECT email FROM relay_recipients WHERE email='%s' AND active='y'

# /etc/postfix/mysql_transport_maps.cf
hosts = 127.0.0.1
user = postfix-read
password = postfixpassword
dbname = postfix
query = SELECT relay FROM transport WHERE pattern='%s' AND active='y'

Before we can now start testing our server, we need to compile these config files to Postfix compatible lookup tables. Do that by running the following command:

$ postmap /etc/postfix/mysql_*.cf

3.2.3. Populate the database

Now we have to fill Postfix’ database with the domains and addresses we’d like to relay. In particular, that means we have to add example.com to relay_domains and transport, and the full addresses to relay_recipients:

INSERT INTO `postfix`.`relay_domains` (`domain` ,`active`)
   VALUES ('example.com', 'y');
 
INSERT INTO `postfix`.`relay_recipients` (`email` ,`active`)
   VALUES ('john.doe@example.com', 'y');
 
INSERT INTO `postfix`.`transport` (`pattern` ,`relay` ,`active`)
   VALUES ('example.com', 'smtp:[mx.my-e-mail-service-provider.tld]', 'y');

The entry structure for each table is different. Please refer to the Postfix manual for details (cp. transport, relay_domains, and relay_recipient_maps).

Note: the database structure above is not optimal since it requires redundant entries in three different tables. Even though the structure is not perfect, I have chosen this layout to make it easily understandable!

4. Test your server

After this short configuration period it’s now time to finally start the Postfix server:

$ /etc/init.d/postfix start

To make sure you didn’t make any mistakes in the configuration, you should now check the log files. Postfix and SQLgrey both use syslog so that you should be able to determine the system’s status like this:

$ tail -n 20 -f /var/log/syslog

If the log doesn’t show any errors, we can now try if everything works as expected. To do so, simply connect to the server from your home computer via telnet:

$ telnet relay.example.com 25
Connected to relay.example.com.
Escape character is '^]'.
220 relay.example.com ESMTP Postfix
HELO somebody
250 relay.example.com
MAIL FROM: some@address.tld
250 2.1.0 Ok
RCPT TO: john.doe@example.com
450 4.7.1 <john.doe@example.com>: 
   Recipient address rejected: Greylisted for 5 minutes

If Postfix replies with a 450 error code, i.e., relay temporarily denied, everything works just fine. On the server side, the log should output something like this:

postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
 
sqlgrey: grey: new: 4.3.2(4.3.2.1), some@address.tld->john.doe@example.com 
postfix/smtpd: NOQUEUE: reject: RCPT from 1-2-3-4.your-isp.tld[4.3.2.1]: 
    450 4.7.1 <john.doe@example.com>: 
    Recipient address rejected: Greylisted for 5 minutes; 
    from=<some@address.tld> to=<john.doe@example.com> ....
 
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]

Wait 5 minutes and try connecting again via telnet. This time, SQLgrey will detect that this is your second delivery attempt and add the sender e-mail and its IP address to the automatic white list (AWL). Postfix will accept your mail and forward it to your provider’s mail server (according to the transport-table):

postfix/smtpd: connect from 1-2-3-4.your-isp.tld[4.3.2.1]
 
sqlgrey: grey: reconnect ok: 4.3.2(4.3.2.1), 
    some@address.tld -> john.doe@example.com (00:22:42) 
sqlgrey: grey: from awl: 4.3.2, some@address.tld added 
 
postfix/smtpd: client=1-2-3-4.your-isp.tld[4.3.2.1]
postfix/cleanup: message-id=<201001...@relay.example.com>
postfix/qmgr: from=<some@address.tld>, size=422, ...
postfix/smtp: to=<john.doe@example.com>, 
    relay=mx.my-esp.tld[12.34.56.78]:25, status=sent, ...
postfix/qmgr: removed
 
postfix/smtpd: disconnect from 1-2-3-4.your-isp.tld[4.3.2.1]

5. Go live: change the DNS record

Play around a little and make sure that everything works as expected. If it does, change the DNS record like described above, i.e., set the MX record of the domains to be relayed to your server’s IP address.

Note: For the first few mails, you should definitely watch the logs. If anything goes wrong, you can always change back the MX record. But be aware that DNS changes might take up to 48h!

If you have any questions, please comment below. I am open for suggestions!

On my journey through the Web I looked into the details of various business notebooks. I stumbled across the Dell Latitude E6400, which looks nice and has everything I wanted. Unfortunately, it also seems to have problems with Ubuntu. In particular, the fan seems to run all the time (and not too slow, but very loud) – as many forum posts and user reviews prove [1,2,3,4].

Since I couldn’t find a solution, I decided to ask the Dell support via their support chat.

Update Nov/09: It appears that a BIOS update addresses the heat problem. Please refer to the community forum post (thanks to chato) for details.

Here’s what I found out. Note: This is a chat with the German Dell support from September 21, 2009, translated into English.

Me:
The Latitude E6400 has (like many forum posts say) a big problem with Ubuntu. In particular, the fan is always on and makes a lot of noise.

Agent:
Welcome to the Dell Chat.

Me:
Hallo. I’d like to know if there already is a solution for this problem. I was thinking about buying a E6400, but I couldn’t find any solutions for the fan/noise problem in Ubuntu.

Agent:
Since we do not support Ubuntu on this device, Dell does not work on a solution. Ubuntu is currently only supported on netbooks. There is no solution.

Me:
Okay. Do you think there will be a solution in the future? Is Dell planning to support Ubuntu on the E6400? Or never?!

Agent:
As I said, we don’t sell the device with Ubuntu.

Me:
Okay. Thank you for the honest answer. I’m gonna have to buy a Thinkpad then … Have a nice day.

Agent:
I’m sorry. But we don’t support Linux.

Me:
It’s not your fault. Thanks anyway. Good bye.

Agent:
I’m so sorry.

So the outcome is: The Dell Latitude E6400 doesn’t support Ubuntu and is NOT planning to solve the fan/noise problem.. That means, I’m gonna have to buy one of those ugly Thinkpads. But what other solutions are there?! None!

Even though this is a good idea for some situations, most of the times, it’s just annoying: Collecting ideas for seminar papers or a thesis, for instance, is almost impossible without being able to Copy & Paste certain paragraphs from the PDF.

Fortunately, Linux can solve this problem with a simple tool called pdf to text. This command line tool simply strips all text from the PDF file and saves it to a given text-file.

Installation

The tool is part of the package poppler-utils and can be installed via your favorite package manager, e.g. apt-get:

$ apt-get install poppler-utils

Extract text from PDF files

This is also pretty simple and the man-page gives the instructions: pdftotext [options] <PDF> [<text-file>].

$ pdftotext PDF-file-with-copy-and-paste-restriction.pdf

In case you’d like to perform this for every PDF-file in a folder (recursive search), simple do that:

$ find -name '*.pdf' -exec pdftotext "{}" \;

After executing the command, there will be a *.txt-file for each PDF file in the folder, – containing the plain-text of the corresponding PDF file.

This is where a password manager comes in handy: Open the password vault by typing in the master password, put in all you secrets and crucial information, save it and be happy. As if!

Almost every password manager I found on the Web was crowded out by details so that it took minutes to add a single account. What I wanted was something like a text-file with password — and that’s what I made: A simple command-line password safe.

How it works

Simsafe is nothing else but a simple Perl script wrapped around the symmetric encryption functions of GPG. Every box with GPG and Perl installed can hence use the script (= every linux box!). And even if the simsafe-script is not installed on the system, a simple gpg –decrypt FILE shows the plain text contents. That is, Simsafe files are nothing more but text-files with password!

Usage

It’s simple and really easy to use.
Syntax:

$ simsafe FILE

Create a new password safe file

$ simsafe mySafe
simsafe: Creating a new password safe 'mySafe'
simsafe: Please enter the new password: (type in your password)
simsafe: Please confirm the password: (confirm your password)
simsafe: Executing editor vi ...
  // VI or your favourite editor opens
  // and you can edit the plain text file
simsafe: Encrypting with GPG ...

Adding/Removing entries to the password vault

$ simsafe mySafe
simsafe: Please enter the safe password: (type in your password)
simsafe: Decrypting with GPG ...
simsafe: Executing editor vi ...
  // VI or your favourite editor opens
  // and you can edit the plain text file
simsafe: Safe unchanged.

Get it!

It’s a reeeaallly small script and this post is possible longer than the perl-file itself, but here it is. Simply download it, make it executable and put it somewhere in your PATH.

Download: Simsafe v0.1, Apr. 2009

The Kademlia Protocol

Kademlia is based on four simple remote procedure calls (RPCs) and guarantees a very fast and lightweight exchange of information by using the connection-less UDP instead of TCP. Every node as well as every entry of the DHT is tagged with an identifier, called node ID or key. By XORing identifiers, one can calculate the distance between them and is hence able to walk through the network knowing whether one is near or far to a specific node. This so called XOR metric makes it possible to group the entries of the DHT around the K closest nodes to the entry’s key. An entry with the key 10, for instance, will be stored at the nodes with a node ID close to 10.

Kademlia + PKI = KadS: The Secure Extension

In my Bachelor thesis, I am going to extend Kademlia from a simple peer-to-peer protocol to a trusted and secure network. The proposed KadS network is almost identical to the Kademlia network, i.e. it consists of the described RPCs and implements the same XOR metric. The major extension to the protocol is that every node is equipped with a public/private key-pair signed by a trusted CA. This extends the normal Kademlia network to a public key infrastructure (PKI) in which every communication is encrypted, every node can be trusted and only verified nodes can participate in the network.

Every connection is synchronously encrypted with a session key which is created and exchanged in a handshake procedure when two nodes first meet. That is, a node has a different session key for the connection with each node it knows in the network. The handshake is similar to the one used in other software except for the fact that both clients exchange their public key and need to verify each others identity. After a successful handshake, two nodes are able to exchange lightweight encrypted messages using the session key and the basic protocol applies

An application: The distributed URL blacklisting service

As an application to the proposed KadS network, I chose a distributed URL blacklisting service. Please read more in my Bachelor thesis exposé, or simply download the whole thesis:

Blacklisting Malicious Web Sites using a Secure Version of the DHT Protocol Kademlia.

As it brings its benefits, cost savings and new customers, every new technology also comes with the more or less known downsides. Even if IT managers are qualified to consider most of the details in how to use and implement them, new software, hardware or resources will – no matter what – always cause unpredicted problems. Due to the IT dependence of today’s companies, every downtime, bug or system overload of a production system directly results in declining profits and higher costs. Especially for service providers, every downtime is business critical to many dependent companies and has to be prevented.

Therefore, companies spend a considerably high amount of money and time to create a stable, flexible and extensible IT environment that supports their business by minimizing risks, increasing availability and allowing to provide better service levels to customers.

Virtualization is a key technology that addresses to achieve these goals. It allows to run multiple virtual computers on the same physical system. By creating an abstraction of the underlying hardware, it allows to execute a variety of virtual machines (VMs) on top of a virtualized hardware.

This article will discuss how the technology of virtualization works, what advantages it offers and why it is an essential part of today’s data centers. The focus will be the server virtualization solution VMware Infrastructure, the flagship product suite of VMware Inc.

1. Evolution of Data Centers and Software Deployment

Traditionally, each big enterprise – be it an IT company or anything else – had their own IT department, their own IT infrastructure and therefore also a data center where all required services such as e-mail, SAP and Web servers were hosted. Over the last decade, the ongoing trend of outsourcing has changed a lot in the modern IT landscape. Companies continuously try to eliminate cost generating areas and outsource them to external service providers.

1.1. Software as a Service

Not only many data centers, but also applications and services are operated and administered by so called application service providers (ASPs). Unlike the classic approach where software has to be purchased, licensed and installed on company owned servers, “in the Software as a Service (SaaS) model, the application or service is deployed from a centralized data center across a network [..] providing access and use on a recurring fee basis” (SIIA, 2001). That is instead of generating uncompensated costs by running and maintaining required non-revenue generating services, one can concentrate on the core business and leave updates, security fixes and high availability issues to the service provider.

1.2. The Impact of Virtualization on SaaS Providers

While service providers are expanding their data centers, more and more companies reduce their IT to a minimum and “rent” many applications from ASPs. Hence, service providers have to shoulder thousands of different applications and operating systems (OS) in their own data center. Each of them has to be available 24/7 and adjustable to the customer’s wishes. Managing data centers of this size is very expensive and requires special technologies to be operated efficiently. Server virtualization, i.e. the hosting of many different guest OSs on one physical system, is a key technology for SaaS providers and has a major impact on their profitability. Due to the fact that virtualization reduces the number of required physical servers, a virtualized data center can minimize the hardware costs while at the same time allowing a flexible way of distributing resources to customers. Multi-tenant applications are designed to serve many customers at the same time, but often do not include the functionality to flexibly distribute available server resources such as memory or processor time. Virtualization implicitly includes the ability to assign resources according to any kind of rules. Service providers can for instance base resource allocation according to their pricing model and hence attract any kind of customer.

Moreover, virtualization allows SaaS providers to easily provide image-based virtual servers on the one hand, but makes it also possible to customize an application or operating system elaborately on the other hand. That is, service providers can for instance charge a small amount of money for a standardized virtual server with few processor power, and charge a higher price for adjustable systems with more CPU power.

On the application side, many services are designed to serve only one customer or organization. Virtualization makes it possible to run normal single tenant applications as if they were designed for more users without having to redesign them. It also can bring other benefits such as highly isolated systems or a better control over the service levels.

In fact, “the benefits are so significant, that [..] no SaaS provider will be able to be competitive without using virtualization.” (Parallels, 2007)

Since Unison can also be used to sychronise more than two hosts, it’s perfect for big amounts of data that has to be shared in a team.

A scenario like this is possible and works for me: UserA <-> Server <-> UserB.
But of course, also other users could sync with the server. Unison rocks!

Today, after reinstalling his OS, my friend got the following error message:

Warning: inconsistent state.  
The archive file is missing on some hosts.
For safety, the remaining copies should be deleted.
  Archive are96968da50882488164ef52510703a8e on 
     host <UserAs-local-hostname> is MISSING
  Archive ar664775fc717afcf6cc46edbc47d25641 on ...

The easy solution

Unison keeps archive-files for each pair it synchronises with. One on each host, i.e. one on your local PC (UserA) and one on the server, both stored in a folder called ~/.unison and looking like the files above.

The easiest solution would be to just delete all archive-files on the server as unison suggests. But that would result in a situation where all the other users that sync their files with the server would have to do a slow sync which is extremely slow and sometimes takes more than 20 minutes for large folders (e.g. > 10 GB).

The best solution …

… is to just delete the required archive-files on the server. Pretty easy, that’s how it works:

Log on to the server (assuming that it’s a Linux machine, but it certainly works similar for any other system) and find the relevant archive files:

$ ssh mysyncuser@mysyncserver.com
$ cd ~/.unison
$ fgrep '//UserAs-local-hostname/' ar*

You should get something like this:

Binary file ar664775fc717afcf6cc46edbc47d25641 matches

Delete the listed files and it should work like a charm.