Hello :)
I decided today - since I have a little time - to write about the importance of following well defined and established processes when defining technical solutions for the clients.
Coincidentally, I have been involved recently in give back activities to the Architecture Profession within the company I work for. In essence, it is a program where a selected group of people act as SME/Chief Architects for the participants (the assigned Solution Architects responsible for technical solution creation), guiding them through the process of delivering an architectural solution to a given client problem. The purpose of the program is to help the participants gain real world experience through means of applying the theoretical knowledge they gained while following Architecture courses, thus acquiring the skill set required to successfully deliver architectural solutions on their own.
In short - since this kind of work takes place internally at my company therefore I won't give a lot of details - they need to do the selection of a delivery process from the RUP based framework we have in place, customize it based on the project conditions (and create work breakdown structure) , and then proceed to follow it by creating the artifacts that describe the solution. All this effort would take place under our supervision as Chief Architects.
This exercise described above, reminded me of the importance of following these delivery processes while creating the technical solutions proposals for the clients.
For instance, we would need to organize ourselves and decide to follow the TSD delivery processes while working on the technical solution proposals; employ AD 2.0 for example while executing the projects, once the contract is secured; and do TDAs each step of the way (for technical solution proposal, as well as for the delivery part).
Of course, in order to be able to achieve all that we would need structural organizational changes to support these activities within our center, we would have to train architects, teach the method, employ technical solution managers.
But the flip side of all that is that we would gain the ability at an organizational level be able to respond to these kind of opportunities in a much coherent way, and therefore increase our changes as an organization to win new contracts - and equally important, deliver them successfully.
sâmbătă, 5 aprilie 2014
duminică, 9 martie 2014
Future for DevOps
I thought I would share a few words about DevOps. As we all know, DevOps is a crossover between Development and Operations. It's about the development, collaboration and continuous integration, deployment, monitoring of applications, which are common activities that usually take place in an IT project.
And the future of these activities is, you guessed it: in the Cloud :). It looks like not only the applications are moving to the Cloud, but also the development and deployment activities.
IBM is already working on an array of tools in this area, aimed at covering the development, collaboration and deployment of software and applications. The codename is BlueMix (www.ibm.com/bluemix) and it will make it much easier to deploy your code in the Cloud. It also has tight integration with JazzHub for the development life cycle, including software development processes.
BlueMix will offer features such as Application Compositions Environments for creation and mgmt of services, Runtimes and Frameworks support, Middleware Services. Operational and Application services.
More specifically, some features are listed below:
In conclusion, with BlueMix in the Cloud you will be able to run your apps, develop through JazzHub integration, use provided APIs and Services to help jump start your development, use Cloud integration features to connect to on premises systems, or to other public and private Clouds, or expose your own API. Or use SaaS SDKs.
So why would we need BlueMix over the current Cloud development PaaS offerings, including application servers, databases webservers and so on?
-- Because it will make it much easier to connect to other applications to share, sync, and manage data through a common Data SDK, do API Integration (i.e. tap into SaaS), do Mobile app development through the provided support, setting up a productive development environment, do monitoring - or run the code on the runtime of your choice.
In conclusion, soon you will be able to do development, collaboration, reuse and connect to other services and use runtime environments of your choice - all in the Cloud. You will be able to focus more on building business applications thus shortening your time to market, and focus less on setting up your development, testing and production environments ;)
A new era is upon us: Welcome to the new IBM Cloud Development Platform!
And the future of these activities is, you guessed it: in the Cloud :). It looks like not only the applications are moving to the Cloud, but also the development and deployment activities.
IBM is already working on an array of tools in this area, aimed at covering the development, collaboration and deployment of software and applications. The codename is BlueMix (www.ibm.com/bluemix) and it will make it much easier to deploy your code in the Cloud. It also has tight integration with JazzHub for the development life cycle, including software development processes.
BlueMix will offer features such as Application Compositions Environments for creation and mgmt of services, Runtimes and Frameworks support, Middleware Services. Operational and Application services.
More specifically, some features are listed below:
- Runtimes such as Liberty, Node.js, Ruby;
- DevOps such as Monitoring and Analytics, Git Hosting, Deployment Automation, IDE of choice (even web based, in the same way MS offers Azure Visual Studio);
- Data Mgmt Services such as SQL & JSON databases, MapReduce;
- Mobile Services: Push, CloudCode, Mobile app mgmt
- Web application services such as Data & Session cache, elastic MQ, SSO, Log analysis and more.
In conclusion, with BlueMix in the Cloud you will be able to run your apps, develop through JazzHub integration, use provided APIs and Services to help jump start your development, use Cloud integration features to connect to on premises systems, or to other public and private Clouds, or expose your own API. Or use SaaS SDKs.
So why would we need BlueMix over the current Cloud development PaaS offerings, including application servers, databases webservers and so on?
-- Because it will make it much easier to connect to other applications to share, sync, and manage data through a common Data SDK, do API Integration (i.e. tap into SaaS), do Mobile app development through the provided support, setting up a productive development environment, do monitoring - or run the code on the runtime of your choice.
In conclusion, soon you will be able to do development, collaboration, reuse and connect to other services and use runtime environments of your choice - all in the Cloud. You will be able to focus more on building business applications thus shortening your time to market, and focus less on setting up your development, testing and production environments ;)
A new era is upon us: Welcome to the new IBM Cloud Development Platform!
vineri, 14 februarie 2014
Java Exceptions Management, stacktrace logging and checked versus unchecked
This subject seems to be rather straightforward, correct? :)
I have been involved during the last weeks in situations connected to this subject, and I thought I would share my thoughts on the matter.
At the first sight, one might think that programming around java exceptions could be considered tedious, since you more or less have to handle them or delegate their handling to the calling software layers.
I would think otherwise. I am of the opinion that the Java exceptions have been designed well originally. While slightly verbose, their proper handling could greatly ease the maintenance, bug finding and monitoring of java programs.
I have experienced this first hand; while working with software written in such a way that prevented proper management and logging of java exceptions, I found that it is much harder to find the errors in the programs written by others - without knowing the code base, it was difficult to fix bugs or alter the existing code - I often had to revert to old style debugging. It shouldn't be that difficult - one look on the stacktrace usually should be enough to understand the situation.
On the checked versus unchecked exceptions subject. I found that it usually is difficult to predict _many_ of the situations that might appear while the program is running, and this is due to the inherent complexities of the conditions and situations that might appear; For this reason alone, I rather prefer to use checked exceptions as much as possible, that could be handled in a way during the program execution. This approach usually allows the program to recover from additional previously unconsidered situations, possibly with the user help.
One example is the NullPointerException: while unchecked, it might appear in many situations. Some of them, usually, could be corrected by the user itself, so it might be worth giving the user a chance to do so. For example, in case of missing (config) files and others. Therefore, it might be worth handling this exception in some cases within the java program itself, possibly by employing the usage of checked exceptions instead.
I have been involved during the last weeks in situations connected to this subject, and I thought I would share my thoughts on the matter.
At the first sight, one might think that programming around java exceptions could be considered tedious, since you more or less have to handle them or delegate their handling to the calling software layers.
I would think otherwise. I am of the opinion that the Java exceptions have been designed well originally. While slightly verbose, their proper handling could greatly ease the maintenance, bug finding and monitoring of java programs.
I have experienced this first hand; while working with software written in such a way that prevented proper management and logging of java exceptions, I found that it is much harder to find the errors in the programs written by others - without knowing the code base, it was difficult to fix bugs or alter the existing code - I often had to revert to old style debugging. It shouldn't be that difficult - one look on the stacktrace usually should be enough to understand the situation.
On the checked versus unchecked exceptions subject. I found that it usually is difficult to predict _many_ of the situations that might appear while the program is running, and this is due to the inherent complexities of the conditions and situations that might appear; For this reason alone, I rather prefer to use checked exceptions as much as possible, that could be handled in a way during the program execution. This approach usually allows the program to recover from additional previously unconsidered situations, possibly with the user help.
One example is the NullPointerException: while unchecked, it might appear in many situations. Some of them, usually, could be corrected by the user itself, so it might be worth giving the user a chance to do so. For example, in case of missing (config) files and others. Therefore, it might be worth handling this exception in some cases within the java program itself, possibly by employing the usage of checked exceptions instead.
vineri, 24 ianuarie 2014
Java Web Start - using parameters in URL for the application started via Web Start
A few days ago, I have been faced with the problem of having to pass some parameters (via a URL using a GET request) to a Java Web Start based application, something I haven't done in the past :)
This is actually quite simple, what you basically have to do is to generate the JNLP file dynamically instead of using a static one. There are several ways of generating a JNLP file dynamically, such as using a JSP file, servlet, SOA oriented service etc.
I will shortly present a way to generate this JNLP file by using a JSP. I am assuming you already have a .jnlp file, and you want to adapt it to use parameters.
1. Rename the existing .jnlp file to jsp.
2. Adding the following directives at the beginning of the jnlp, now jsp file (will be used to compose the URL and will set the response headers):
<%
String urlSt = request.getRequestURL().toString();
String jnlpCodeBase=urlSt.substring(0,urlSt.lastIndexOf('/'));
String jnlpRefURL=urlSt.substring(urlSt.lastIndexOf('/')+1,urlSt.length());
%>
<%
response.setContentType("application/x-java-jnlp-file");
response.setHeader("Cache-Control", null);
response.setHeader("Set-Cookie", null);
response.setHeader("Vary", null);
%>
3. Customize the <jnlp> tag codebase param to include your parameters. As you can see, I am using two example parameters :
<jnlp codebase="<%=request.getScheme() + "://"+ request.getServerName() + ":" + request.getServerPort()+ request.getContextPath() + "/" %>" href="jnlp/launch.jnlp?url=<%=request.getParameter("url")%>&user=<%=request.getParameter("user")%>">
That way, the params you send using the GET request, will be passed to the application started via Web Start.
This is actually quite simple, what you basically have to do is to generate the JNLP file dynamically instead of using a static one. There are several ways of generating a JNLP file dynamically, such as using a JSP file, servlet, SOA oriented service etc.
I will shortly present a way to generate this JNLP file by using a JSP. I am assuming you already have a .jnlp file, and you want to adapt it to use parameters.
1. Rename the existing .jnlp file to jsp.
2. Adding the following directives at the beginning of the jnlp, now jsp file (will be used to compose the URL and will set the response headers):
<%
String urlSt = request.getRequestURL().toString();
String jnlpCodeBase=urlSt.substring(0,urlSt.lastIndexOf('/'));
String jnlpRefURL=urlSt.substring(urlSt.lastIndexOf('/')+1,urlSt.length());
%>
<%
response.setContentType("application/x-java-jnlp-file");
response.setHeader("Cache-Control", null);
response.setHeader("Set-Cookie", null);
response.setHeader("Vary", null);
%>
3. Customize the <jnlp> tag codebase param to include your parameters. As you can see, I am using two example parameters :
<jnlp codebase="<%=request.getScheme() + "://"+ request.getServerName() + ":" + request.getServerPort()+ request.getContextPath() + "/" %>" href="jnlp/launch.jnlp?url=<%=request.getParameter("url")%>&user=<%=request.getParameter("user")%>">
4. Add your arguments again to the application description tag inside your JNLP file (now JSP):
<application-desc main-class="com.ibm.myapp.SomeGreatApp">
<argument><%=request.getParameter("url")%></argument>
<argument><%=request.getParameter("user")%></argument>
</application-desc>
5. Deploy your app, then you can access the JNLP file using a request such as:
http://localhost:8080/mayapp/jnlp/launch.jsp?url=http://localhost:8080/someURL/somepage.html&user=some_username
That way, the params you send using the GET request, will be passed to the application started via Web Start.
luni, 20 ianuarie 2014
Big Data Analytics in Cloud - Patterns and Use-Cases
Introduction
Nowadays, people have implemented several Big Data Analytics solutions already, and through these, a few macro patterns have emerged. You can find them below, along with a short explanation for each of them.
I also mentioned a few implementation possibilities next to them. Of course, there are other options as well to each of my examples (I decided to mostly refer IBM software).
1. Landing zone warehouse (HDFS -> ETL DW)
This is composed of a landing zone(Big Data Warehouse), handled using InfoSphere BigInsights (based on Hadoop), that reads the data from various sources and stores it on HDFS. This could be done through ETL batch processes. This data is unstructured/semi structured or structured. It generally needs to be processed and organized.
From there, data can be loaded into a Big Data Report Mart, via batch ETL. The main advantage is that the data becomes structured and organized according to customer needs. This could be Cognos, Netezza, DB2 etc.
From there, data can be loaded into a Big Data Report Mart, via batch ETL. The main advantage is that the data becomes structured and organized according to customer needs. This could be Cognos, Netezza, DB2 etc.
It could then be queried though SQL queries or by making usage of reporting features of the above tools.
2. Streams dynamic warehouse (RT Filter -> HDFS/DW)
The data in this pattern is organized mostly the same as the one above. It too has a Landing Zone (Big Data Warehouse) handled with InfoSphere BigInsights for instance, and a Big Data Summary Mart (handled with IBM PureData For Analytics for instance). From this Summary mart, reports could be extracted using SQL for instance.
The main difference is that the incoming "data stream" into the cloud is going to be initially processed using InfoSphere Streams on real time. It will be filtered and analysed, then stored in the Big Data Warehouse (landing zone). This data is going to be mostly structured. A part of it is going to be stored directly into the Big Data Summary mart.
The upside in this case is that while the data mart is going to contain structured information as well, the processing happens in real time as the data comes, with the help of InfoSphere Streams. Therefore the Data Warehouse will contain real time structured data.
3. Streams detail with update (combination 2 & 1)
This is a combination between the first two. The data streams are going to be real time processed as they come, using InfoSphere Streams. They will be then stored on both Landing Zone and Big Data Summary mart as in #2. Additionally, there will be also a Detail Data Mart, loaded through ETL processes from the Landing Zone (Big Data Warehouse). This way additional processing could be done on this data which requires analysis over large data sets for instance, or maybe by making use of data that is not available in real time.
The applications will access both the Big Data Summary mart loaded with data processed real time, as well as the Detail Data mart, filled by the ETL batch processes.
4. Direct augmentation (HDFS -> augment -> DW)
The incoming data will be loaded in the a Big Data Warehouse (Landing Zone) via batch ETL. Contains mostly unstructured data. It could be then directly accessed though Hive (via a Virtual database) from HDFS.
Additionally, there is also an existing Data mart containing existing transactional data acquired previously, from other sources (internal perhaps).
Applications will make use of both landing zone and data mart simultaneously.
5. Warehouse augmentation (HDFS -> augment analytics)
This is basically the same as #1, with the addition that the data acquired in the Big Data Warehouse is going to be subsequently enhanced using an Analytics Engine. It will be then loaded to the Summary Data mart, using the same ETL batch processes.
6. Streams augmentation (augment & filter RT -> HDFS/DW)
This patterns also makes use of the Analytics Engine, that will enhance/augment the data coming to the InfoSphere Streams (and processed in real time). From there, the filtered data will be saved to the Big Data warehouse, handled by InfoSphere BigInsights. The ETL batch processes will read and process this data, and will load it to the Data Summary Mart. It can be then accessed though SQL queries for instance.
7. Dynamic cube (HDFS - > search indexes)
This is the most complex pattern of all :). The data virtually arrives in the Big Data Warehouse handled by InfoSphere BigInsights for example.
It is subsequently being processed by an index crawler, indexed by making usage of a big data index, and then accessed through a Virtual Data Mart.
They say is that though this pattern you're building your own google (search engine) ;)
1. Big Data Exploration - find, visualize and understand the data to improve decision making.
2. Enhanced 360 degree view of the Customer - all customer data in one place by incorporating all sources.
3. Security/Intelligence Extension - monitor and detect in real time.
4. Operations Analysis - analyze a variety of machine and operation data for improved business results.
5. Data Warehouse Augmentation - integrate big data and data warehouse capabilities for improved business results. Optimize your data warehouse to enable new types of analysis.
It is subsequently being processed by an index crawler, indexed by making usage of a big data index, and then accessed through a Virtual Data Mart.
They say is that though this pattern you're building your own google (search engine) ;)
Primary Big Data Use-Cases
1. Big Data Exploration - find, visualize and understand the data to improve decision making.
2. Enhanced 360 degree view of the Customer - all customer data in one place by incorporating all sources.
3. Security/Intelligence Extension - monitor and detect in real time.
4. Operations Analysis - analyze a variety of machine and operation data for improved business results.
5. Data Warehouse Augmentation - integrate big data and data warehouse capabilities for improved business results. Optimize your data warehouse to enable new types of analysis.
joi, 16 ianuarie 2014
A few words on Clouds - Introduction
2013 was the year where the clients have embraced the cloud solutions, and several cloud patterns have crystallized and were validated through successful clients engagements.
There are several cloud solutions offerings, including open source. Examples come from Google, Microsoft, Amazon, IBM, Rackspace, open source (OpenStack) and the list continues...
What are the benefits of using a cloud? It's a way to externalize the tasks/jobs of hardware/software administration, maintenance and support by delegating these to other entities along with all the requirements that need to be solved through these activities (hosting, provisioning, availability, security, scalability, monitoring, etc). It also might provide access to third party software using SOA.
On cloud usage patterns, a few approaches have been explored. One could have public clouds where cloud services are being offered to external clients, or private clouds used by singular entities.
The public clouds have been diversifying their offerings, and a few usage patterns have emerged:
- IaaS or Infrastructure as a Service. It's basically a way to make computing power available to clients. One common way to achieve this is through virtualization software, where the clients get access to virtual images where they could install/configure software of their choice. Depending on the business case, this could be actually cheaper for clients because they are freed from buying and administering the hardware and OSes. Through agreed SLAs, the clients usually know what to expect regarding the services they receive.
- PaaS or Platform as a Service. The provider delivers a computing platform, where programming language execution environments, web servers and databases are available. This builds on top of IaaS by providing the additional software required to build and run applications.
- SaaS or Software as a Service. In addition to PaaS services, the clients are provided access to application software and services as well. This might be in the form of SOA or other related services, and/or perhaps third party software such as Microsoft Office, Exchange and others. The clients are able to deploy they applications to the cloud, make use of cloud infrastructure and platform, deployed software, as well as any SOA related end points made available by the cloud provider.
We will talk about Patterns for Big Data & Analytics in Cloud next
There are several cloud solutions offerings, including open source. Examples come from Google, Microsoft, Amazon, IBM, Rackspace, open source (OpenStack) and the list continues...
What are the benefits of using a cloud? It's a way to externalize the tasks/jobs of hardware/software administration, maintenance and support by delegating these to other entities along with all the requirements that need to be solved through these activities (hosting, provisioning, availability, security, scalability, monitoring, etc). It also might provide access to third party software using SOA.
On cloud usage patterns, a few approaches have been explored. One could have public clouds where cloud services are being offered to external clients, or private clouds used by singular entities.
The public clouds have been diversifying their offerings, and a few usage patterns have emerged:
- IaaS or Infrastructure as a Service. It's basically a way to make computing power available to clients. One common way to achieve this is through virtualization software, where the clients get access to virtual images where they could install/configure software of their choice. Depending on the business case, this could be actually cheaper for clients because they are freed from buying and administering the hardware and OSes. Through agreed SLAs, the clients usually know what to expect regarding the services they receive.
- PaaS or Platform as a Service. The provider delivers a computing platform, where programming language execution environments, web servers and databases are available. This builds on top of IaaS by providing the additional software required to build and run applications.
- SaaS or Software as a Service. In addition to PaaS services, the clients are provided access to application software and services as well. This might be in the form of SOA or other related services, and/or perhaps third party software such as Microsoft Office, Exchange and others. The clients are able to deploy they applications to the cloud, make use of cloud infrastructure and platform, deployed software, as well as any SOA related end points made available by the cloud provider.
We will talk about Patterns for Big Data & Analytics in Cloud next
joi, 9 ianuarie 2014
Why we need Architects
Nowadays one could see numerous attempts to break down and conquer the complexities of software development through processes, roles, standards and tasks; while I think this is certainly a valid approach, I also believe that it's no silver bullet.
No matter how smartly you're going to design your processes, tasks and roles, your weak link is going to be your employees nevertheless. Therefore, it makes sense to invest in some people to attempt to strengthen this link. You need experienced and capable people, and with the right attitude. In addition to that, I also believe that there are a few key roles that most IT projects need and these are: PM, Architect, Team Lead, Developer and Tester (as you all know, there are other roles as well, but not important for the sake of our argument here). Of course, sometimes, in the real life, some people will have more that one role.
While the PM needs to make sure the project is within time and budget, TL's job is to coordinate the development team through the daily tasks and do development of its own, the Architect's job is to make sure the project goals are being met technically.
It is the Architect's job that there is a clear understanding with the client on what needs to be built, to go to the drawing board and make a technical plan on agreement with the client (gathering feedback from everyone along the way), and to make sure that the artifacts delivered at the end of the project meet those objectives.
There is a wide agreement nowadays on the fact that most of the IT project fail. Failure means in this context that most projects go actually over time and/or budget. The most important reason for this is that the teams involved fail to meet the technical challenges along the way, being IT requirements, designing the solution, estimations, development, testing and so on. This happens most of the time not because they might not have the desire to do the job, but because building computer systems is hard :).
That is why you need an experienced technical person in your project that is concerned mainly with the overall technical goals of the project (Architect/Technical Manager). This includes making sure that the requirements are in line with client interests and plans, that they are sound, achievable, clear and non conflicting. It also includes the technical solution design and plan, agreed with the client. It includes the proactiveness in identifying and taking action to address the technical risks that may appear in the project, as well as making sure that the project deliverables are in line with the technical solution plan.
It only makes sense that the presence of Architects in those teams would improve the chances of success for projects. And as we all know, success means happier clients, more business from them and more money for us.
More on this subject later, have fun
No matter how smartly you're going to design your processes, tasks and roles, your weak link is going to be your employees nevertheless. Therefore, it makes sense to invest in some people to attempt to strengthen this link. You need experienced and capable people, and with the right attitude. In addition to that, I also believe that there are a few key roles that most IT projects need and these are: PM, Architect, Team Lead, Developer and Tester (as you all know, there are other roles as well, but not important for the sake of our argument here). Of course, sometimes, in the real life, some people will have more that one role.
While the PM needs to make sure the project is within time and budget, TL's job is to coordinate the development team through the daily tasks and do development of its own, the Architect's job is to make sure the project goals are being met technically.
It is the Architect's job that there is a clear understanding with the client on what needs to be built, to go to the drawing board and make a technical plan on agreement with the client (gathering feedback from everyone along the way), and to make sure that the artifacts delivered at the end of the project meet those objectives.
There is a wide agreement nowadays on the fact that most of the IT project fail. Failure means in this context that most projects go actually over time and/or budget. The most important reason for this is that the teams involved fail to meet the technical challenges along the way, being IT requirements, designing the solution, estimations, development, testing and so on. This happens most of the time not because they might not have the desire to do the job, but because building computer systems is hard :).
That is why you need an experienced technical person in your project that is concerned mainly with the overall technical goals of the project (Architect/Technical Manager). This includes making sure that the requirements are in line with client interests and plans, that they are sound, achievable, clear and non conflicting. It also includes the technical solution design and plan, agreed with the client. It includes the proactiveness in identifying and taking action to address the technical risks that may appear in the project, as well as making sure that the project deliverables are in line with the technical solution plan.
It only makes sense that the presence of Architects in those teams would improve the chances of success for projects. And as we all know, success means happier clients, more business from them and more money for us.
More on this subject later, have fun
Abonați-vă la:
Postări (Atom)