Implement custom sampling for AppInsights logs to control logging costs

Problem: AppInsights log volume can get very expensive very quickly for a production environment. We'd like an optional ability to sample non-essential logs.

Use case: We'd like to sample a percentage of noisy trace and dependency logs (info type traces and successful dependencies) while keeping all request and error logs, as well as important trace and dependency logs (i.e. audit logs (trace) and failed dependencies).

Tried but didn't help: AppInsights SDK provides log sampling for Java apps via Sampling Overrides. However, these work in a all or nothing mode, meaning that we cannot selectively sample traces and dependencies while keeping request logs if the said traces and dependencies are logged under the same request span/context. To sample traces and dependencies, we'd have to sample requests as well, which is not what we want.

Proposed solution:

• An optional config per service that allows certain log types to be randomly sampled. This should be completely optional, meaning no breaking change and logging works exactly the same if a service doesn't explicitly opt-in for sampling.
• A service, if decides to opt-in, will provide a sampling percentage for different log types. For the initial implementation, we shall start with info and dependency sampling only. We can later extend the same principal to other tables.
• A service only needs to modify its application.properties file to opt-in for sampling.
• Even if sampling is enabled for a service for traces, audit logs (which are logged in the trace table) should not be sampled.
• Even if sampling is enabled for a service for dependencies, failed dependencies should not be sampled.

Action: MR !16 (merged)