Automated-Data-Analytics Product Development

Home / Analytics / Automated-Data-Analytics Product Development

In today’s competitive world, analytics has become must for continuous development. Nowadays, data is available from different data sources. People are analyzing it and making use of it in decision making. Some organizations are finding data insights and providing it to targeted audience. Many data analytics product are being developed.
If you have good source of data, data-scientist are able to find useful insights from it. Then you can make use of that data for development. In today’s world, data is available either in static form or in dynamic (real-time). Data is either structured or unstructured. This data can be analyzed using different BI tools. But in some cases data need to be processed through different steps and then data insights can be found. In order to do this processing, Data-Analytics products are being developed.

We will see below – product development structure/methodology:
Data
Data Scientist
Research  Software Engineer (Data)
Role
(Research)
(Architect)
(Software-Engineer)
(Data-Analyst)
Analysis and Research
Solution-Design
Development &Testing
Validation
Flow
 
Research-Up-gradation
Research-Up-gradation
Changes
Development lifecycle of ‘Automated-Data-Analytics’ product

 

 

Skills required to play each role: At every step one must aware about data analytic skill.
Research:
  1. Sound knowledge about data domain and future.
  2. Mathematical skill and graph analysis.
  3. Data analytics and insights.
Architect:
  1. Problem solving skills.
  2. Knowledge about data domain and future changes (if available)
  3. Technology
Software-Engineer:
  1. Technological implementation
  2. Introduction to data domain
Data-analyst:
  1. Knowledge about data domain.
  2. Data analytical skills.
     In waterfall-model or big products this type of roles should be performed by different individuals and it’s an efficient way for development.
If evolutionary model is followed then this kind of work distribution can cause overhead and at that time work should be distributed among Data-Scientist and Research-Software Engineer. In evolutionary model requirements are changing frequently. If we had work distributed across different phases then it will add communication overhead and change overhead at each phase. If minimum phases/stages are involved then error due to communication propagation will be reduced. In minimum phases/stages communication can be transferred effectively.
Here, research software engineer has to play major role in product development phase. One has to understand research thoroughly. If research methodology is accurate but programming it as it is may become difficult/critical. In product development improvement and maintenance are must to do task in future. Hence development should be flexible to accommodate changes further as well as programming research methodology accurately. Hence ‘developer’ has to consider research and development with same weightage. One should design solution which can accommodate both the things hand in hand. Hence ‘Research software engineer’ must have good understanding of research as well as problem solving skill. Also as a role of software engineer one should have programming, testing and development life cycle skill.
 We can say Research-Software-Engineer is one who works next to Data scientist in innovative product development. As demand of Data-scientist is growing day-by-day, it suggests demand of Research-Software-Engineer will also increase.

 

Technologies/Knowledge for data analytics:
  1. Logic
  2. Set-theory
  3. Graphical analysis
  4. Introduction to statistics
  5. SQL
  6. Scripting languages – Python, awk, shell
  7. Programming languages
  8. Big-Data : Hadoop, SPARK, Hive
  9. R
  10. Linux
  11. Performance engineering
  12. Problem solving.
    And many more coming each day…………