Vodafone UK Company

  Vertex AI Vision is an AI-powered platform to ingest, analyze and store video data. Vertex AI Vision lets users build and deploy applications with a simplified user interface. Using Vertex AI Vision you can build end-to-end computer image solutions by leveraging Vertex AI Vision's integration with other major components, namely Live Video Analytics, data streams, and Vision Warehouse. The Vertex AI Vision API allows you to build a high level app from low level APIs, and create and update a high level workflow that combines multiple individual API calls. You can then execute your workflow as a unit by making a single deploy request to the Vertex AI Vision platform server. Using Vertex AI Vision, you can: 1. Ingest real-time video data. 2. Analyze data for insights using general and custom vision AI models. 3. Store insights in Vision Warehouse for simplified querying and metadata information. Vertex AI Vision workflow: The steps you complete to use Vertex AI Vision are as fo...

  CDF offers an extensive set of data aggregation and analysis tools in a single package. The functionality of these tools is central to some of Cybervision's ongoing projects. CDF pipelines can be either batch or real-time. Batch applications can be run manually, based on time schedule, or another trigger. Real-time applications run continuously, obtain data as it becomes available, and process it immediately. This flexibility is important for efficiently handling tasks of various nature. Let's take a look at an example CDF pipeline. Assume that raw, semi-structured medical data is loaded into Cloud Datastore. Our goal is to validate data ranges and dates, normalize the records by renaming certain fields and aggregate them by patient ID and date. In the end, the aggregate data is loaded into BigQuery so it can be used by analytics tools for further exploration and analysis. CDF comes ready with tools to do the necessary data manipulations. Cloud Data Fusion enables users to bu...

Training pipelines let you perform custom machine learning (ML) training and automatically create a 'Model' resource based on your training output. Before you create a pipeline Before you create a training pipeline on Vertex AI, you need to create a 'Python training application' or a 'custom container' to define the training code and dependencies you want to run on Vertex AI. If you create a Python training application using TensorFlow, scikit-learn, or XGBoost, you can use our pre-built containers to run your code. If you're not sure which of these options to choose, refer to the 'training code requirements' to learn more. Training Pipeline options:  Training pipelines let you perform custom machine learning (ML) training and automatically create a 'Model' resource based on your training output. Before you create a pipeline Before you create a training pipeline on Vertex AI, you need to create a 'Python training application' or a ...

The access controls for Cloud Bigtable can be configured at the project, instance, or table level. At the project level, you can do the following: 1. Allow a user to read from any table in any instance of the project but not write to those tables. 2. Allow a user to read from and write to any table in any instance of the project. 3. Allow a user to manage any instance within the project. At the instance level, you can do the following: 1. Restrict a user to be able to read from development but not production instances. 2. Allow a user to read and write to development instances and read from production instances. 3. Allow a user to manage development instances but not production instances. At the table level, you can do the following: 1. Allow a user to read from a table but not write to the table. 2. Allow a user to write to a table but not read from the table. Cloud Bigtable has permissions that allow access to resources, such as instances, application profiles, clusters, an...

There are several ways to control access to Cloud Storage resources, including buckets and objects in those buckets. 1. Cloud IAM is the preferred way to control access to buckets and objects. 2. For complex access control logic or when you need to control access to individual objects, you may need to use access control lists (ACLs). 3. Signed URLs is another option for granting access. These URLs are generated by you and shared with someone to whom you want to grant access but only for a short period of time. 4. If you want to control what can be uploaded to a bucket, you can use a signed policy document. In this section, we will focus on the use of Cloud IAM with Cloud Storage. Cloud Storage permissions are organized around resources, such as buckets, objects, and Hash-based Message Authentication Code (HMAC) keys. The bucket permissions allow users to create, delete, and list buckets. There are also permissions for getting and updating metadata as well as setting and getting IAM pol...

1. Designing for Security and Compliance. Considerations include: 1. Identity and access management (e.g., Cloud IAM). 2. Data security (encryption, key management). 3. Ensuring privacy (e.g., Data Loss Prevention API). 4. Legal compliance (e.g., Health Insurance Portability and Accountability Act (HIPPA), Children's Online Privacy Protection Act (COPPA), FedRAMP, General Data Protection Regulation (GDPR)). Since data engineers work with diverse sets of data, they will likely need to use a variety of data stores that use access controls. They also should be prepared to work with sensitive data that needs additional protections. This chapter introduces several key topics of security and compliance, including: 1. Identity and access management. 2. Data security, including encryption and key management. 3. Data loss prevention. 4. Compliance. We'll begin with identity and access management, because it is fundamental to many security practices. Identity and Access M...

When you create a virtual (VM) on Compute Engine, you specify a machine series and a machine type for the VM. Each machine series is associated with one or more CPU platforms. If there are multiple CPU platforms available for a machine type, you can select a minimum CPU platform for the VM. A CPU platform offers multiple physical processors, and each of these processors are referred to as a core. For all processors available on Compute Engine, a single CPU core can run as multiple hardware multithreads through Simultaneous multithreading (SMT), which is known on Intel processors as Intel Hyper-Threading Technology. On Compute Engine, each hardware multithread is called a virtual CPU (vCPU). When vCPUs are reported to the VM as occupying different virtual cores, Compute Engine ensures that these vCPUs never share the same physical core. The machine type of your VM specifies its number of vCPUs, and you can infer its number of physical CPU cores using the default vCPU per core ratio for ...

I will describe in this article the machine families, machine series, and machine types that you can choose from to create a virtual machine (VM) instance with the resources you need. When you create a VM, you select a machine type from a machine family that determines the resources available to that VM. There are several machine families you can choose from and each machine family is further organized into machine series and predefined machine types within each series. For example, within the N2 series in the general-purpose machine family, you can select n2-standard-4 machine type. All machine series support preemtible VMs, with the exception of the M2 machine series: 1. General-purpose: best price-performance ratio for a variety of workloads. 2. Compute-optimized: highest performance per core on Compute Engine and optimized for compute-intensive workloads. 3. Memory-optimized: ideal for memory-intensive workloads, offering more memory per core than other machine families, with up to...

An instance group is a collection of virtual machine (VM) instances that you can manage as a single entity. Compute Engine offers two kinds of VM instance groups, managed and unmanaged: 1. Managed instance groups (MIGs) let you operate apps on multiple identical VMs. You can make your workloads scalable and highly available by taking advantage of automated MIG services, including: autoscaling, autohealing, regional (multiple zone) deployment, and automatic updating. 2. Unmanaged instance groups: let you load balance across a fleet of VMs that you manage yourself. Managed instance groups (MIGs) Use a managed instance group (MIG) for scenarios like these: 1. Stateless serving workloads, such as a website frontend. 2. Stateless batch, high-performance, or high-throughput compute workloads, such as image processing from a queue. 3. Stateful applications, such as databases, legacy applications, and long-running batch computations with checkpointing. Compute Engine maintains each of th...

  In this article, we are going to see about what is Instance groups and how you can use instance groups to manage a group of virtual machine instances. Instance group allows your applications to gracefully handle increase in traffic as well as it reduces the cost when the need for the resource is lower. And we will see about types of instance groups and the features of instance groups in this article. What is Instance Groups? Instance group is nothing but a group of virtual machine instances managed as a single entity. You can manage groups of similar VMs having similar life cycle as one unit and you can create two types of Instance group in Compute Engine. They are: 1. Managed instance group. 2. Unmanaged Instance group 1. Managed Instance group (MIG) These are identical virtual machine created using instance template. So these VMs, which are part of managed group will have same machine type, same image and the same configuration. Some of the features that are provided for the ma...