The Data Scientist’s Toolbox

SAS Certified Base Programmer for SAS 9

R Programming

Learning Hadoop

Exploratory Data Analysis

AWS Certified Cloud Practitioner

Master of Science (M.S.)

Bootstrapped a startup in the observability pipeline space and pivoted to building an AI assistant for DevOps teams leveraging LLMs. Build POCs using Vector, Redpanda, DIFY, OpenAI, Python. 

Did market survey, met platform vendors, and potential customers to understand the market.

Founder

Generative AI software development. As an Independent technology consultant, helped a 2-year-old startup, to bring their POC to production readiness – Python code review & fixes, establish CI/CD pipeline, create K8s deployment for local development and deployment on AWS, upgrading OpenAI models (davinci-002->gpt-4-turbo), evaluating and switching out TTS engine (Mozilla TTS -> Eleven labs).

Python Tech Lead / System Architect

Angel Investor in industry leading Network-based threat detection and response.

Investor

RiskIQ was acquired by Microsoft for $500 million in July 2021, and I became a full-time employee of Microsoft in Oct 2021.

As a founding Director of the Collections Team, my team were responsible for collecting internet scale data for all RiskIQ products. The tech stack included Java, Hadoop, Kafka, AWS – S3, EC2, MSK, RDS. I introduced data-quality metrics to improve the recency metrics of critical threat intelligence asset class from 80 to 90%.

I was given the additional charge of Data Engineering Team; with this, I was responsible for feature development and operations on in-house MapReduce based data pipelines(100s) running on on-prem MapR Hadoop, Hive and Spark, processing peta-bytes of information required for up-to-date threat information critical for our customers.

I also doubled the size of the combined team through rigorous hiring to help us though the Microsoft acquisition process and coming new product development. Established "continuous onboarding" as a team practice 

Director of Data Engineering

Lead development engineer for Emerging Threats suit & Threat Intelligence products. 

• Lead developer of a worldwide IoT network to collect millions of cybersecurity threat events per day in partnership with major security appliance vendor. (Python/Django, Ansible, PostgreSQL, Redis)
• Lead developer for building data pipeline for malware related threat intel data that is used for new cybersecurity threat research, detection and mitigation. (Java, Kafka, EC2, EFS, S3, RDS, Prometheus, Grafana) 
• Early Lead developer to develop AWS Lambda functions to process threat signatures & logs (AWS Lambda, Python, RDS, Cloudwatch)
• Core Team member for Emerging Threats Threat Intelligence data products (Scala/Java, Play framework, Redshift, Hadoop, Pig, Kafka, PostgreSQL) 
Recognition: CEO’s quarterly recognition awards – 3 times.

Staff Software Engineer

Hands-on Principal Engineering Manager with experience in leading team of engineers to build data driven, industry leading big data solutions, in RiskIQ division of Microsoft Security.

As Principal Engineering Manager of Platform Collections and Data Engineering team, my team and I were responsible for collecting internet scale threat intelligence data and to build complex MapReduce data pipelines(Java; 100s) on a large data lake (20+Petabytes) on big data platform (Hadoop, Hive, Spark). 

I managed a team of 12 engineers including 2 engineering managers reporting to me to deliver data powering the RikskIQ product portfolio as well as two groundbreaking Generally Available products on Azure – Microsoft Defender for External Attack Surface Management (MDEASM) and Microsoft Defender for Threat Intelligence (MDTI).

Spearheaded the migration of a large-scale, operationally intensive data collection infrastructure from on-prem data center, and AWS to the Azure platform – HDI (first party 

Principal Software Engineering Manager

Founding software engineer at a building energy consumption & accounting startup. Instrumental in getting continued angel funding for the business.

• Developed Intelligent Building Energy consumption monitoring and messaging platform.
• Developed software and hardware to read instantaneous energy values from power meter, store it for analysis and alerting. (C, MODBUS, PostgreSQL, Redis, Python, Lua)
• Developed software to visualize energy consumption of individual and aggregate meters. Show summary statistics of kWh, dollar amount, consumption per sq.ft etc., (Python/Django, SQLAlchemy)
• Implemented a mathematical and computation intelligence based solution to profile and predict building energy usage. (Python and SQL)
• Generate alert messages and dispatch to building managers. (Python, SMS) 
• Developed of web portal for building managers to visualize and manage their energy profile online in near-real time. (Python, Javascript, amcharts)
• Developed analyst driven, semi-automated

Software Architect

Software Development

Principal

Distributed Systems | Data Engineering | AI

github

Know about Pradeep Gowda's work experience, education, bootcamps, certifications, and much more. Pradeep Gowda