Is there a difference between a SOA repository and a SOA registry?

Generally speaking a registry is a location that stores information about registered entities. A few of examples of registries could be a town's birth registry which records all births in the town, a College class registration office, and an annual article index of a publication which lists all articles that appeared in the publication during a particular year.

Registry is defined in the Webster's College Dictionary as the "place where a register is kept", and the dictionary defines register as "a book in which records of events, names, etc., are kept".

In the examples provided above we see that the birth registry will contain a list of all names of people who were born in the town, their dates of birth and maybe some other information like the address at the time of birth, ages of parents, names of parents, etc. We do not expect to see the actual people in the registry ! The people will be in their offices, homes or wherever else.

Similarly, in the case of the annual article index, which is another form of registry we expect to see the titles and dates of publication of articles that appeared in the publication during that year. We do not expect to find the articles themselves in the index of articles.

A repository on the other hand is a location where actual artifacts are stored. We can guess that for the example of the birth registry, the actual repository would be hospitals, delivery clinics and places where people give birth. This is where we can actually find the babies.

Similarly in the case of the article index, the actual articles would be find in the various issues of the publication throughout the year, and would be stored in a library.

Repository is defined in the Webster's College Dictionary as " a place where things are deposited, stored or offered".

So now in the context of SOA, we can define the Service Registry and Service Repository as follows.

Service Registry
A comprehensive list of all SOA services available to an enterprise to achieve a business objective. The list would include the service identification information, location of the service, service classification (purpose), and information about how to interact with the service.

Service Repository
A storage location that houses the SOA service and its components, is involved in controlling access to the service and the presentation of the service to its users

If we ponder for a moment we notice that some key areas of a SOA service are neither part of a service registry or service repository - at least by defintion. For example a service's history, statistics about the the service, performance reports, etc.. by definition are not part of the repository. Some SOA implementations or frameworks might decide to store such service management reports and information in the repository, others might wish to develop a SOA Service Management Repository which will house that kind of informaton.

Systemic Reflections from the Obama - McCain Race

There is no doubt that Obama not only won the elections, but also smashed his opposition. Looking at the Electoral votes we see a torrent gap 349/163 and a popular vote of 53%/46%... what a surprise... totally unpredictable... I personally thought he might win 51%/49% and maybe an electoral vote ratio of 1.1/1... not 2.14/1..

So this tells us that human systems are totally unpredictable.. engineering a system with a human as part of it might be suicidal in some cases. So lets take the example of engineering a commercial aircraft, is the pilot part of the system or not? Most system engineers will say.. not really ... the system is the plane (structure, mechanical, electrical, etc..) we can model those, but we can't model the pilot, what if the pilot is too slow to land, or fell asleep during take off.. so many soft factors to model, so lets take the pilot out of the picture.. and then we test (verify) out plane with an average pilot... that is a pilot who did not have a bad day, did not lose a loved one the week before, is not tired, has no spousal problems, etc.. So the test passes, aircraft declared ready for production, sold and used for years, without a hitch...

We recall the Egypt Air flight than mysteriously dropped out of the Eastern US skies on Halloween night 1999, until our date we have theories and thoughts of why the sky bus blew up and perished in the air... some say the pilots walked out of the cabin to stretch, when the senior pilot walked in and committed suicide, other theories are cracks in the wing, a third fuel leakage, and a fourth contribute the root cause to a strike 10 yrs ago when that batch of 747s where in Seattle on the assembly line, and that because they were not in a good mood the quality of manufacturing standards was below nominal.. who knows.. it could be all the above, or none of the above.. bottom line is that many of these root causes include a human factor.. totally unpredictable .. just like the American voters.. the time has come to include soft factors into engineering problems -- Point 1... we can't ignore soft factors

Now, wasn't it the McCain campaign that distributed those millions of DVDs in swing states to tell them about how radical Islam is? Covering up truth, and smearing others does not work anymore... the people are more educated nowadays than the 80s, 70s, and 60s.. thanks to the so called Internet, and its myriad of facebook, you tube, google and countless other applications.. --- Point 2... Fooling customers never works, eventually they will know that you are marketing a 747 and selling a prop.

Finally, end users buy and use what they want, not what you tell them they need... Americans don't want their daughters and sons in Iraq, Americans don't want their 401Ks wiped out, they don't want to lose their jobs, telling them that pulling out of Iraq is not only stupid, but irrelevant.. regardless whether it is a good idea or not, its not what they want to hear. End users want to hear solutions to their problems, they don't want solutions to other problems, and they are not smart enough to know where root causes come from, you where there when the plane crashed.. then you must be involved. --- Point 3... It doesn't matter how brilliant or great your idea is, if it does not solve my problem, its worthless in my perspective [1], also I don't care why the system crashed, the point is that you did not do a good job designing it. Don't blame the storm, blame the plane [2]

We can wish as much as we want, model as much as we want, take the means, but at the end whatever Allah ordains will happen...

Now whether Obama will deliver on his promises is a completely different story, history will let us know.

SOA Programming Allows Independence

The SOA programming model is to embrace underlying programming languages that are already deployed in the enterprise. It does not mater whether applications are .NET, CICS, Java, SAP based, or based on any other technology, these technologies are all transparent to SOA.

SOA allows the architect to hide any variations or differences between languages and programs to maximize reuse. Additionally we can drill down into unique aspects of a specific language or program only when needed, without exposing the details.

One of the major benefits of this programming model followed by SOA is to program and develop business and service design rather than a technology. For example the programming focus would be on a student grade verification process, rather than a .NET session.

Any experiences to share?

XML's role to SOA

SOA leverages the standard XML data representation to reduce underlying complexity of application environments. Some are examples are;

- Passing XML documents and schemas between applications allows for the standardization of data format and type, which this improves predictability and performance.

- XML documents are easily understood by different stakeholders such as architects, analysts, developers and users, leading to enhanced data maintenance, traceability and clarity.

- XML allows the support of standardized vocabularies across the enterprise, hence reducing mis-interpretation of corporate data and data models.

- XML allows a SOA based system to realize a foundation data representation and data management layer.

For SOA to leverage the benefits of XML, a solid XML architecture should be developed and implemented. Such architecture should include architectural components for data transport, structure, validation, verification and modeling. Accommodations for RPC-style messaging needs to also be considered if an environment does not support document-style messaging (SOAP). In such cases we need to ensure no conflicts exist between usage of RPC-style SOAP messages (from shaping XML documents around a parameter data exchange model).

A Slotting System for Misery, Poverty and Crime

It was a bit of a shock to see that Marylanders favored bringing slot machines to Charm City, and its suburbs. It is no more than a call to bring crime, prostituition, and long term economic failure. This is not just my opinion as a Muslim Systems Engineer, but that of any person who has some basic knowledge of economics and social reform will have the same opinion.

What comes easy goes easy. Slots brings a system of vapor to Maryland. No true productivity, no true growth and no true value proposition, other than greed to the operators.

We should not be surprised to see more bankruptcies, shattered households, and chaos similar to the global financial crash pretty soon in the Free State.

Should Systems Engineers Study Psychology?

Given the fact that systems engineers deal with systems, and most systems comprise of a human element in it, it seems to make sens that systems engineers need to know a little about how people act.

Humans are very complex system components, they can be viewed as a system in themselves (human system), or even a system of systems (SoS) (digestive, neural, circulatory, psychological, muscular, etc..)

So maybe after all those biology classes we used to study in middle school do have some benefit to enterprise architecture. I guess systems engineers who are involved in soft system should study not only psychology, but maybe also other human systems of interest to the largest SoS.


I came across an interesting article at http://www.techjournalsouth.com/news/article.html?item_id=6299 which discusses cognitive systems engineering, which is defined as, "A combination of psychology, anthropology, computer science, design and systems engineering, the field of cognitive systems engineering is the understanding and designing of systems that require human intellectual work".

Got to go dig those science books..

So What are the Services in SOA?

Service-oriented architecture has gained heavy adoption over the past few years. The next several postings will provide a high level SOA introduction. I am hoping the posting could serve as a basic primer for systems and project professionals working on a SOA project.

What is service-oriented architecture?
An architecture that allows the alignment of business processes and objectives to Information Systems in a flexible and adaptive manner, through the defintion of services.

What are these services?
Services are modular and repeatable business processes. A service could be defined at different levels of the organization. for example, a "customer update" could be a service that allows a customer to update their profile. On another level, a subscriber authentication process - to allow customer to gain access to her profile to update it - could be a service.

What are the main characteristics of a SOA service?
Services are also loosely coupled, maintaining a relationship that minimizes dependencies and only requires the awareness of connected services. Services are stateless and minimize information related to a specific activity.

So some examples of simple services in a sales department of a business could be; "collect customer information", "collect product /service information", collect payment terms". Each of these services could be broken down further as needed

Good Engineers are Managers and Leaders

Many schools of thought separate between leadership and management. While I agree they are different they are intertwined constructs. Read more here.

Lets take a simple example, one of that of a sales manager. Most people will agree that a sales manager is primarily a management role. However in the case of a small travel agency serving the air travel needs of its clients, the sales manager on a daily basis finds herself performing tasks such as establishing new relationship with potential suppliers (leadership), negotiating contracts and deals (leadership), motivating her sales staff (leadership), developing sales target plans (management), performing forecasts (leadership), reviewing sales reports (management), training employees (management(, coaching high school interns (leadership), addressing customer complaints (management) and calculating sales team commission (management). This is another example of how a sales manager role in a small business deploys leadership skills to remain highly competitive and ensure superior client value.

How much of your time as a systems engineer do you spend managing, leading and engineering?

Perfection Never Ends... KISS brings us closer

Last night I was chatting with a friend of mine who is working on the launch of a new project. I asked, "How far are you from launching?" His unexpected reply was "We will miss this year's launch and do it next year, I want to make sure everything is complete and people get to say wow"

The question we need to consider "Is it better to launch today when there is need, but have a modest service offering?, or to spend another year perfecting the service offering and miss the boat on this year's demand?"

KISS (Keep it Simple & Short) brings us closer to deployment and reality.. apply agility... get something working out the door, serve the need, use the opportunity to make further improvements in later releases... perfection never ends.

The Assertive Systems Engineer

So you are on a project to help the team develop a successful system. You notice things that don't make sense, or are not they way they should be. You raise your concern, "Hey, team we need a process to automate testing". Everyone looks at you as if you are some wacko, and then moves on with their business. Two weeks later and after 4 more team meetings you raise the concern again, and again. Finally one team member says "We always did testing this way, and we don't have the money or time to develop the automation, so don't keep repeating this".

What do you do? Shut up? Go with the flow? or be stubborn, resist to work in a chaotic environment? Actually the reaction should be neither. Smart systems engineers need to be adaptive, flexible and understanding of the business constraints and technical limitations. They also need to be assertive and counselors of best practices and approaches.

Document your technical opinion, the problem as you see it, the proposed solution, the rationale for the solution and consequences of not addressing the problem. Be assertive about your technical opinion, and also flexible to adjust the solution as needed to meet the laws of physics and constraints the team must abide with.

Remember systems engineering is all about engineering a system with the most optimum means, in accordance to sound decisions and rationale

ISO IEC 15288 / INCOSE SEBoK v3.0 Life Cycle Stages

Life-cycle definition is an important aspect of any system. By defining a life-cycle we can establish a framework for meeting stakeholder needs in an orderly fashion.

This order comes from the presence of stages in the life-cycle, and decision-making at the end of each stage to determine the readiness to move on. Stakeholders are concerned with the business case, funding and capability of a system at each stage.

INCOSE SEBoK ver 3.0 defines seven life-cycle phases. Each of these phases are listed below. Phases 2 - 7 are the same as the six phases defined in ISO 15288.

Pre-concept Phase:
Inputs: New ideas
Process: Creative systems engineering and requirements definition
Outputs: Identified enabling technologies

Concept Phase:
Inputs: Studies, experiments, models
Process: Requirements identification (if not done in pre-concept), indepth feasibility studies, prototypes, risk evaluation, early validation
Outputs: System requirements, feasible design solution

Development Phase:
Input: System requirements
Process: Development, integration, verification and validation
Outputs: Developed, verified and validated system

Production Phase:
Input: Tested system
Process: Modifications, re-verification, cost reduction
Output: Manufactured product

Utilization Stage:
Input: Performance reports, change requests, operational work orders
Process: Operations, change management
Output: Operational product, upgrades

Support Stage:
Inputs: Modifications, change requests
Process: Change control, maintenance, cost reduction, support
Output: Sustainable service and supported mission

Retirement Stage:
Inputs: change requests, competing systems
Process: System migration, removal, disposal
Output: Disposed system

System Sustainability: No Longer an Option

System developers rarely analyze or even worry about the long term sustainability of a system. The main focus is usually on today's and the near future's needs. The client says "I need to consolidate these two billing systems by next January".

Everyone on the projects starts to focus on this requirement. It becomes an architectural driving requirement, functions and operations of the new system will be based on this requirement and dependencies will start appearing, which if traced will be found to have largely come from that main requirement - consolidate by January.

No one gives thoughts to what happens after January. How and what will the new consolidated system look like and need to do to remain sustainable and not turn obsolete in a year or two and require another consolidation project with something else at the time.

Last year I was fortunate to do some research work under the supervision of Dr. Peter Sandborn of the University of Maryland. We looked into this topic - how to create sustainable software - and have come up with a model that might help extend the life-cycle of software systems.

With news that countries like Oman will run out of oil in 40 years popping up, sustainability in all aspects is no longer an option, but a requirement.

Agile In a Nutshell

Agile development is all about short iterations. Using what is known as a scrum process requirements are iterated and turned into use cases which are tested and so on.

The goal in agile is to test early, to identify early defects; and use continuous stakeholder feedback to deliver high-quality consumable code through use cases, and a series of short time-boxed iterations.

A New Buzzword in Town - Cloud Computing

So first it was Application Service Providers (ASPs) then Utility Computing and now comes Cloud Computing... It seems that every couple of years the industry needs a new set of buzzwords to keep the PR machine running.

Well there might be some slight differences between utility and cloud computing, but I would not really identify them as differences, I would just note them as the same with a slightly different scope. Instead of renting computing resources - utility computing - cloud computing rents applications and IT services.

So what is this cloud computing after all? Reading through various articles it seems to be nothing more than an application service provider hosting enterprise IT services for its clients. Its just technology improved from what was available in the mid-nineties, and vendor are looking for a way to get over the unsuccessful connotation of ASPs by relabeling the concept into cloud computing.

Some links on the topic from various trade rags for those interested:

http://gigaom.com/2008/02/28/how-cloud-utility-computing-are-different/
http://www.businessweek.com/magazine/content/07_52/b4064048925836.htm
http://www.gartner.com/it/products/research/cloud_computing/cloud_computing.jsp
http://www.technologyreview.com/Infotech/19397/?a=f
http://www.businessweek.com/technology/content/nov2007/tc20071116_379585.htm

What is RUP?

RUP short for Rational Unified Process is a process for software engineering. Originally develped by Rational Software, Inc. - now acquired by IBM.

RUP focuses on the following best practices:

1. Four major phases in a software development cycle (inception, elaboration, construction and transition)

2. Within each phase RUP allows for iteration and incremental development. this supports refined requirements, risk reduction and results oriented management.

3. Disciplines which group activities by nature. There are nine disciplines defined (business modeling, requirements, analysis and design, implementation, test, deployment, configuration management, project management and environment). These disciplines spread across the four phases of a software project.

4. Manage requirements using tools that support traceability, prioritization, validation and quality management.

5. Develop component-based architectures, where components are primary concepts for design. They are self-contained modules with clear interfaces that can provide a well-defined function.

6. Visually model software using languages such as UML, or SysML. Numerous tools on the market support these languages such as IBM's Rational Suite, IBM's Telelogic DOORS, iRise, Visual Use Case, Visual Paradigm and many others.

7. Verify software quality, especially in the areas of functional, performance and reliability compliance.

8. Control changes to software through a traceable and end-to-end approach.

An Enterprise as a System of Systems

Sometimes it hard to identify whether a system is really a system of systems (SoS) or just a system. INCOSE defines seven challenges that a system engineer could use to identify a SoS [1]. I find two of them of particular value. The first test condition is that each system can operate independently. The second is that each system has a different life-cycle.

Looking at an enterprise we can see that in a well optimized, stream-lined and modular enterprise (as if this exists!) , the various business units or divisions could operate independently. Also we notice that the various programs, services or products offered by the enterprise have different life-cycles.

Requirements: Engineering vs. Management

Which is a subset of the other, is the process of requirements management a subset of requirements engineering or the other way around?

To best answer this question lets get back to the basics. What is "engineering" and what is "management".

The term engineering is used to indicate an activity whereby an engineer utilizes findings of sciences to realize solutions which solve problems. To provide an example, a society which has a problem crossing a river asks a civil engineer to realize a solution to this problem. The engineer based on his readings and input from scientists in the fields of chemistry, physics and materials suggests using a steel structure which can withstand corrosion from the environment, and hot summer temperatures as well as extremely low winter temperatures. The engineer performs calculations to detemine bridge loads and how the structure will behave to cross winds and othe environmental elements.

So back to requirements engineering, it is the process used to determine how to best engineer (recognize, analyze and synthesis) a requirement.

Generally speaking management is defined as the activity of controlling, organizing, planning, and directing. It can be argued that planning and analysis do overlap and hence both requirements engineering and management overlap to some degree, and I would agree with this. However requirements management has a much broader scope than engineering, requirements management plans, organizes, controls and directs the processes related to requirements "handling", this handling includes engineering, development, maintenance and obsolecence. Changes to requirements are considered part of requirements maintenance which is part of reqs mgmt.

Requirements management involves developing a database to store requirements, a report from this database could be the RTVM, a list of configurable items (CIs), requirements management could also include tracing requirements to business requirements upward and to test and verification results downwards, it also ensures that the synthesis process in requirements engineering is sound and consistent and traceable to project work packages in the work breakdown structure (WBS) and eventually the statement of work (SOW). It involves establishing requirements baselines, negotiation with stake-holders and prioritization of requirements and changes.

We should not assume requirements management to be a phase in the system life-cycle, but rather a continuous process overseeing all the phases of the system, nor is it a subset of requirements engineering but rather a superset of the various system phases. This is illustrated in the chart below.

A few papers related to requirements management are listed,

[1] Ian Somerville and Pete Sawyer, "Viewpoints: Principles, Problems and a Practical Approach to Requirements Engineering", Lancaster University, 1997.

[2] James Martin, "A Process for Requirements Definition and Architecture Definition", Texas Instruments.

[3] James Martin, "Systems Engineering Guidebook: A Process for Developing Systems and Products", CRC Press, 1997.

Daring and Leading

A good engineer is one who is daring and leading the stakeholders towards a solution. Younger engineers tend to be more risk taking, which is not always bad. As Pearl Buck once stated "The young do not know enough to be prudent, and therefore they attempt the impossible -- and achieve it, generation after generation."

Daring and risk taking is not wrong and should never be, as long as we manage it. Thats where the whole idea of risk management comes in. Risk management and innovation work hand in hand, and will produce exceptional results if led correctly.

As Eudora Welty once said, "All serious daring starts from within", a good engineer is one who loves what is being worked on, what is being innovated and what is being risked.

Allah (glory be to him) states in the Quran, "Indeed Allah will not change what has been bestowed on a group of people, until they change what is inside of them".

It all starts in the heart and the passion to lead and succeed.

SOA in a Nutshell

Service-oriented architecture has emerged as the means to improve business intelligence, innovation and agility, through developing loosely coupled distinct automation logic units. These loosely coupled automation logic units allow an abstraction of business logic and technology.

SOA encourages units of logic to exist autonomously yet not isolated from each other. In SOA the units of logic are known as "services".

Automation Logic

The automation logic components used by SOA comprise of:

1. Messages: which represent the data required to complete some or parts of a unit of work.

2. Operations: which represent the logic required to process messages to complete a unit of work

3. Services: which represent logically grouped set of operations capable of performing related units of work.

4. Processes: which contain and define the business rules that determine which service operations are used to complete a unit of automation. Instances of processes wherein a group of services follow a particular path through the process logic to complete a task are known as activities.

SOA Principles

Key principles in SOA are:

1. Autonomous services
2. Loosely coupled services
3. Services abstract underlying logic
4. Composable services
5. Formal contracts to define information exchange between services
6. Reusable services
7. Stateless services
8. Discoverable services

Software Components

Systems engineers define requirements at various levels, the highest of those levels is the business level, then comes the enterprise level, then the system level, then the component level. For a physical system a component could be easy to identify, how about software?

One way of defining a component in the software world is an active object with clear self-defined interfaces [1]. Active objects are objects with have their own thread of control.

Another definition is that a component is an object written to specification. Components are defined to meet several criteria namely; multi-use, non-context-specific, composable, contained and of independent versioning [2].

So a component level requirement is a requirement that tells the developer the needs that the code shall be able to accomplish.

[1] Hassan Gomma, "Designing Concurrent, Distributed, and Real-Time Applications with UML", Addison Wesley, 2000

[2] http://en.wikipedia.org/wiki/Software_componentry

Applying the Learning-Teaching Style Process to Jump Start Your Novice Engineers

Working as an elementary school teacher is not an easy job. One of the biggest challenges facing an educator is activating and sustaining the student's motivation.

Richard Lavoie in his book "The Motivation Breakthrough" discusses a four-step process which has been effective with children during challenging times. The process is summarized as follows:

1. Do it for the student - By explaining to the student what needs to be done, we then ask questions about next steps. Using common sense and logic the student should be able to explain what needs to be done to solve the problem. During this step the student is not asked to do the work himself, but only be responsive to questions, actively listening and asking questions.

2. Do it with the student - In this step the educator allows the student the opportunity to do part or all of the work with the assistance of his coach. Upon the completion of this step the student will be able to perform the task, or solve the problem on his own.

3. Observe the student - In this step the educator observes the student doing the work. It allows the educator to evaluate the process of the problem solution and not just the final result. It is a great opportunity for the educator or coach to offer suggestions, praise, clarifications and guidance. Upon completion of this step the student becomes a master in the problem or task.

4. Have the student do it - The student is completely independent and not supervised during the problem solving or task implementation. It provides the student extra confidence and the ability to innovate and improve the process and skills involved.

As Chief Systems Engineers we can apply these concepts to novice team members or new graduates. New members could shadow experienced systems engineers and achieve step 1 and 2 through direct interactions during the shadowing process. Delegation allows the senior team member to observe the junior team member and finally a complete turn-over allows the realization of the fourth step.

Hello I am Your Systems Engineer. How Can I Serve You?

The last thing a systems engineer wants to ask a client during a requirements gathering process or JAD session is "So what features or capabilities do you need?". If the client knows what they need, there is no reason to hire a systems engineer.... right?

So how can an SE find out what a client needs? Asking for what the problems are is probably the first questions an SE needs to bring up. Identifying the problems, the root-cause for these problems, the severity, history and impact are the questions the clients need to hear. Clients understand very well their problems, they usually have a slight idea or thought on potential solutions, and in many cases don't. But all clients will know what their problems are, or at least will be able to explain the manifestations of a problem.

Systems Engineering - Various Perspectives

We are familiar with civil engineering, electrical engineering, mechanical engineering and the many other engineering disciplines. Engineering generally speaking is the application of science and its concepts to solve society's problems. For example a problem of crossing a river can be solved using the sciences of materials, welding, concrete pouring, etc.. through civil engineering which makes it possible to realize a civil solution. Also we can develop a product that lifts people up into the air and cross the river - called a helicopter - through aerospace engineering which makes it possible to realize an aerospace solution. Similarly, systems engineering attempts to solve problems through the realization of a systems solution using the concepts of systems sciences. One difference in the case of systems engineering from other engineering disciplines is that systems engineerings is not only based on sciences of systems, but also on the art of systems thinking.

Here are some perspectives on what systems engineering is,

1. An Engineering discipline focusing on system realization using well developed scientific approaches such as requirements definition, engineering and management; feasibility and trade studies; system interfacing and integration; technical risk control and management; system optimization; system life-cycle management and system modeling and simulation. Systems Engineering has been offered as a post graduate degree in many institutions around the world, mostly at the Masters level. A handful of Universities also offer PhD degrees in Systems Engineering. I expect in the next 10 years we will see B.Sc. degrees offered in systems engineering as a new engineering branch of study, it could also be offered in specializations or as a minor at the undergraduate level.

2. An Engineering approach which combines both art and science. The science of rigorous well-defined engineering theories, processes and approaches, and the science of problem solving using systemic approaches. The art of systems thinking.

3. A management discipline controlling the total system life-cycle. In this case systems engineering is viewed as the interaction of science, organization and the environment and their impact on the system throughout all phases of its life-cycle. It is also viewed as the application of methods, tools and technologies across the life-cycle of the system products.