책 이미지
eBook 미리보기
책 정보
· 제목 : Agile Machine Learning: Effective Machine Learning Inspired by the Agile Manifesto (Paperback)
· 분류 : 외국도서 > 컴퓨터 > 프로그래밍 > 마이크로소프트 프로그래밍
· ISBN : 9781484251065
· 쪽수 : 248쪽
· 출판일 : 2019-08-22
· 분류 : 외국도서 > 컴퓨터 > 프로그래밍 > 마이크로소프트 프로그래밍
· ISBN : 9781484251065
· 쪽수 : 248쪽
· 출판일 : 2019-08-22
목차
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- How do you get started on a data engineering problem?
- There are so many directions and approaches to go in--how do you pick the right one?
- The importance of establishing measurements and metrics early.
- Daily measurements and metric
- The power of metric driven teams and data driven decision making
- who is the customer? Do they need to be represented? Is the customer the data goal?
- how to establish / calibrate what is possible given the data available and the inferences being asked of the team.
- is this new problem, or a known problem with a known recipe?
- Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
- Building models that are changeable, configurable, extensible, pluggable--how to not redo everything when requirements change
- Adjusting metrics when requirements change
- When to throw things away
- Anticipating change (but not over-anticipating change)
- Ensuring that model-features are testable and agile
- Balancing work priorities. When to break a sprint
- Balancing bug fixes versus new work
- When to make long term, long payoff investments
- Balancing metrics that naturally work against each other (duplicate rate versus overmatch, conversion rate versus time on site)
- The role of test, train and regression data sets in the process; how these can be adapted to new or modified requirements; the importance of keeping track of them and how they were generated
- How to build, manage and develop a data labeling / curation /judgment team; how to set expectations for quality of labels, how to know when your devs should be labeling data (to learn the problem space) and if and when to engage with a labeling team.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Deciding on a product cadence and project rhythm
- How to avoid ML death-marches
- How long do we go down a particular path before we explore another direction
- When to invest in better models/hyperparameters versus when to invest in better features
- How big of an improvement is good enough to ship?
- Using flighting to evade analysis paralysis
- Leveraging your users to deliver working software more quickly (fix the world, observing user behavior and clicks)!
- How to fail fast
- How to optimize for DEVELOPER PRODUCTIVITY (the most important thing in Matt’s opinion) and how not to get fooled by processes which result in shoddy engineering that will come back to haunt you. How to deliver DATA in a predictable manner; when how to migrate schema; the hierarchy of changing things, their associated costs and implied degree of design required (i.e. if you can change it easily you can worry about the correctness of the data model less, but - can you really change it easily?)
- Also - how to state problems in the traditional ML frame (test sets, train sets, dev sets, etc.) without the necessity of an “ML’ solution - you can test a heuristic against your test set!
- Designing business focused metrics--ensuring the right problem is solved
- How to recognize metrics for metrics sake that don’t actually relate to the business, the user, or the engineering progress in solving a problem
- How to work with business people to understand a domain
- Helping business people understand what to expect from an ml project
- Exposing out early work/models to business people before they go into production
- Building a vision
- Using metrics to align cross team/cross division work...the importance of shared metrics
- Important face to face meetings
- The importance of data wallows
- How to do daily standups in a data engineering team
- How and when to communicate more complex discoveries and punt to a data wallow
- Encouraging cross team problem solving and ‘happy accidents’ with all hands demos
- And how to avoid these being a n-1 reporting process for a boss / alternatively, why this might be a good thing Presenting data, data viz skills, power point skills are not optional nice to haves but a key skill area for any journey man developer - this is not just for managers and designers.
- Face to face with your customers
- Face to face with your judges and vendors
- How to ensure the software is working every day?
- Testing in production The role of devops
- How to do “model spikes” to figure out how much of an impact a good model could have (e.g. wizard of oz)
- How to do early abandonment of something that won’t work
- How to know if / how / when to let ICs explore vs exploit
- Models in production with no pressure to get better versus academia models
- What if your delivery is really data, not working software? The role of documentation for processes (e.g. deployment) and algorithmic decisions (we did it this way because this data and this experiment showed this result)
- How to have the right amount of urgency
- Setting audacious goals while maintaining realism
- Communicating to sponsors that ml isn't a magic bullet for everything and has limitations
- How to negotiate / avoid contractual obligations around specific quality measures
- The importance of slack weeks
- Making sure people understand why their work matters and how it fits
- How to make live site sustainable
- Cross geo collaboration The hierarchy of goals (big multi-year, annual, quarterly … daily)
- Best practices for multistage classification systems
- Best practices for testing
- Best practices for deploying
- Best practices for feature engineering Best practices around big data systems and pipelines
- Best practices around ‘I need to fix something fast’
- Best practices around performance sensitive ml
- The power of great labeling: The importance of paying attention to judges and how to make them part of the team
- Pros and cons of ‘defense in depth ml’. (Clean catalog versus relevance filters the catalog)
- Data workflow design
- E.g. don’t drop any data vs each stage ‘improves’ the data and drops earlier representations
- How to analyze the gaps in your model
- How to prioritize what model improvements you go after
- What ml is bad at
- When to just brute force it
- When and how to put humans in the loop
- General architectures that allow any component in your system to ‘phone a human’
- Limits to precision and recall--managing the existence of failure cases
- Always design for mitigation - any feedback can be addressed by a) taking the feedback as an example of a type of error that can be explored, quantified, etc. and b) immediately fixed so that that instance is never a problem
- How to manage data patches (as from the above mitigations) so they don’t become stale and bite you back
- Reuse versus build from scratch
- A great feature engineering idea could come from anyone How to make sure you don’t have a “hub and spoke” model where everyone is dependent on one expert (also, I think, known as the surgeon model)
- Asking the right questions and setting goals in a way that encourages everyone on the team to engage their brain
- Setting vision for the team
- Connecting the team to the organizational needs and strategy
- How to do early abandonment of something that won’t work
- Understanding your dsats
- Revisiting and retuning your metrics
- The benefits of having a competitor (competitive metrics)
- The pros and cons of using competitor tail lights as your ground truth - you will never be better
- Early warning signs that something may be going wrong Common problem patterns and the ml techniques used to solve them
