---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.14.0
  name: aigateways.gateway-operator.konghq.com
spec:
  group: gateway-operator.konghq.com
  names:
    kind: AIGateway
    listKind: AIGatewayList
    plural: aigateways
    singular: aigateway
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - description: The URL endpoint for the AIGateway
      jsonPath: .status.endpoint
      name: Endpoint
      type: string
    name: v1alpha1
    schema:
      openAPIV3Schema:
        description: |-
          AIGateway is a network Gateway enabling access and management for AI &
          Machine Learning models such as Large Language Models (LLM).


          The underlying technology for the AIGateway is the Kong Gateway configured
          with a variety of plugins which provide the the AI featureset.


          This is a list of the plugins, which are available in Kong Gateway v3.6.x+:


            - ai-proxy (https://github.com/kong/kong/tree/master/kong/plugins/ai-proxy)
            - ai-request-transformer (https://github.com/kong/kong/tree/master/kong/plugins/ai-request-transformer)
            - ai-response-transformers (https://github.com/kong/kong/tree/master/kong/plugins/ai-response-transformer)
            - ai-prompt-template (https://github.com/kong/kong/tree/master/kong/plugins/ai-prompt-template)
            - ai-prompt-guard-plugin (https://github.com/kong/kong/tree/master/kong/plugins/ai-prompt-guard)
            - ai-prompt-decorator-plugin (https://github.com/kong/kong/tree/master/kong/plugins/ai-prompt-decorator)


          So effectively the AIGateway resource provides a bespoke Gateway resource
          (which it owns and manages) with the gateway, consumers and plugin
          configurations automated and configurable via Kubernetes APIs.


          The current iteration only supports the proxy itself, but the API is being
          built with room for future growth in several dimensions. For instance:


            - Supporting auxiliary functions (e.g. decorator, guard, templater, token-rate-limit)
            - Supporting request/response transformers
            - Supporting more than just LLMs (e.g. CCNs, GANs, e.t.c.)
            - Supporting more hosting options for LLMs (e.g. self hosted)
            - Supporting more AI cloud providers
            - Supporting more AI cloud provider features


          The validation rules throughout are set up to ensure at least one
          cloud-provider-based LLM is specified, but in the future when we have more
          model types and more hosting options for those types so we may want to look
          into using CEL validation to ensure that at least one model configuration is
          provided. We may also want to use CEL to validate things like identifier
          unique-ness, e.t.c.


          See: https://kubernetes.io/docs/reference/using-api/cel/
        properties:
          apiVersion:
            description: |-
              APIVersion defines the versioned schema of this representation of an object.
              Servers should convert recognized schemas to the latest internal value, and
              may reject unrecognized values.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
            type: string
          kind:
            description: |-
              Kind is a string value representing the REST resource this object represents.
              Servers may infer this from the endpoint the client submits requests to.
              Cannot be updated.
              In CamelCase.
              More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
            type: string
          metadata:
            type: object
          spec:
            description: Spec is the desired state of the AIGateway.
            properties:
              cloudProviderCredentials:
                description: |-
                  CloudProviderCredentials is a reference to an object (e.g. a Kubernetes
                  Secret) which contains the credentials needed to access the APIs of
                  cloud providers.


                  This is the global configuration that will be used by DEFAULT for all
                  model configurations. A secret configured this way MAY include any number
                  of key-value pairs equal to the number of providers you have, but used
                  this way the keys MUST be named according to their providers (e.g.
                  "openai", "azure", "cohere", e.t.c.). For example:


                    apiVersion: v1
                    kind: Secret
                    metadata:
                      name: devteam-ai-cloud-providers
                    type: Opaque
                    data:
                      openai: *****************
                      azure: *****************
                      cohere: *****************


                  See AICloudProviderName for a list of known and valid cloud providers.


                  Note that the keys are NOT case-sensitive (e.g. "OpenAI", "openai", and
                  "openAI" are all valid and considered the same keys) but if there are
                  duplicates endpoints failures conditions will be emitted and endpoints
                  will not be configured until the duplicates are resolved.


                  This is currently considered required, but in future iterations will be
                  optional as we do things like enable configuring credentials at the model
                  level.
                properties:
                  kind:
                    description: |-
                      Kind is the API object kind


                      If not specified, it will be assumed to be "Secret". If a Secret is used
                      as the Kind, the secret must contain a single key-value pair where the
                      value is the secret API token. The key can be named anything, as long as
                      there's only one entry, but by convention it should be "apiToken".
                    type: string
                  name:
                    description: Name is the name of the reference object.
                    type: string
                  namespace:
                    description: |-
                      Namespace is the namespace of the reference object.


                      If not specified, it will be assumed to be the same namespace as the
                      object which references it.
                    type: string
                required:
                - name
                type: object
              gatewayClassName:
                description: |-
                  GatewayClassName is the name of the GatewayClass which is responsible for
                  the AIGateway.
                type: string
              largeLanguageModels:
                description: |-
                  LargeLanguageModels is a list of Large Language Models (LLMs) to be
                  managed by the AI Gateway.


                  This is a required field because we only support LLMs at the moment. In
                  future iterations we may support other model types.
                properties:
                  cloudHosted:
                    description: |-
                      CloudHosted configures LLMs hosted and served by cloud providers.


                      This is currently a required field, requiring at least one cloud-hosted
                      LLM be specified, however in future iterations we may add other hosting
                      options such as self-hosted LLMs as separate fields.
                    items:
                      description: |-
                        CloudHostedLargeLanguageModel is the configuration for Large Language Models
                        (LLM) hosted by a known and supported AI cloud provider (e.g. OpenAI, Cohere,
                        Azure, e.t.c.).
                      properties:
                        aiCloudProvider:
                          description: |-
                            AICloudProvider defines the cloud provider that will fulfill the LLM
                            requests for this CloudHostedLargeLanguageModel
                          properties:
                            name:
                              description: Name is the unique name of an LLM provider.
                              enum:
                              - openai
                              - azure
                              - cohere
                              - mistral
                              type: string
                          required:
                          - name
                          type: object
                        defaultPromptParams:
                          description: |-
                            DefaultPromptParams configures the parameters which will be sent with
                            any and every inference request.


                            If this is set, there is currently no way to override these parameters
                            at the individual prompt level. This is an expected feature from later
                            releases of our AI plugins.
                          properties:
                            maxTokens:
                              description: |-
                                Max Tokens specifies the maximum length of the model's output in terms
                                of the number of tokens (words or pieces of words). This parameter
                                limits the output's size, ensuring the model generates content within a
                                manageable scope. A token can be a word or part of a word, depending on
                                the model's tokenizer.
                              type: integer
                            temperature:
                              description: |-
                                Temperature controls the randomness of predictions by scaling the logits
                                before applying softmax. A lower temperature (e.g., 0.0 to 0.7) makes
                                the model more confident in its predictions, leading to more repetitive
                                and deterministic outputs. A higher temperature (e.g., 0.8 to 1.0)
                                increases randomness, generating more diverse and creative outputs. At
                                very high temperatures, the outputs may become nonsensical or highly
                                unpredictable.
                              type: string
                            topK:
                              description: |-
                                TopK sampling is a technique where the model's prediction is limited to
                                the K most likely next tokens at each step of the generation process.
                                The probability distribution is truncated to these top K tokens, and the
                                next token is randomly sampled from this subset. This method helps in
                                reducing the chance of selecting highly improbable tokens, making the
                                text more coherent. A smaller K leads to more predictable text, while a
                                larger K allows for more diversity but with an increased risk of
                                incoherence.
                              type: integer
                            topP:
                              description: |-
                                TopP (also known as nucleus sampling) is an alternative to top K
                                sampling. Instead of selecting the top K tokens, top P sampling chooses
                                from the smallest set of tokens whose cumulative probability exceeds the
                                threshold P. This method dynamically adjusts the number of tokens
                                considered at each step, depending on their probability distribution. It
                                helps in maintaining diversity while also avoiding very unlikely tokens.
                                A higher P value increases diversity but can lead to less coherence,
                                whereas a lower P value makes the model's outputs more focused and
                                coherent.
                              type: string
                          type: object
                        defaultPrompts:
                          description: |-
                            DefaultPrompts is a list of prompts that should be provided to the LLM
                            by default. This is generally used to influence inference behavior, for
                            instance by providing a "system" role prompt that instructs the LLM to
                            take on a certain persona.
                          items:
                            description: |-
                              LLMPrompt is a text prompt that includes parameters, a role and content.


                              This is intended for situations like when you need to provide roles in a
                              prompt to an LLM in order to influence its behavior and responses.


                              For example, you might want to provide a "system" role and tell the LLM
                              something like "you are a helpful assistant who responds in the style of
                              Sherlock Holmes".
                            properties:
                              content:
                                description: Content is the prompt text sent for inference.
                                type: string
                              role:
                                default: user
                                description: |-
                                  Role indicates the role of the prompt. This is used to identify the
                                  prompt's purpose, such as "system" or "user" and can influence the
                                  behavior of the LLM.


                                  If not specified, "user" will be used as the default.
                                enum:
                                - user
                                - system
                                type: string
                            required:
                            - content
                            type: object
                          maxItems: 64
                          type: array
                        identifier:
                          description: |-
                            Identifier is the unique name which identifies the LLM. This will be used
                            as part of the requests made to an AIGateway endpoint. For instance: if
                            you provided the identifier "devteam-gpt-access", then you would access
                            this model via "https://${endpoint}/devteam-gpt-access" and supply it
                            with your consumer credentials to authenticate requests.
                          type: string
                        model:
                          description: |-
                            Model is the model name of the LLM (e.g. gpt-3.5-turbo, phi-2, e.t.c.).


                            If not specified, whatever the cloud provider specifies as the default
                            model will be used.
                          type: string
                        promptType:
                          default: completions
                          description: |-
                            PromptType is the type of prompt to be used for inference requests to
                            the LLM (e.g. "chat", "completions").


                            If "chat" is specified, prompts sent by the user will be interactive,
                            contextual and stateful. The LLM will dynamically answer questions and
                            simulate a dialogue, while also keeping track of the conversation to
                            provide contextually relevant responses.


                            If "completions" is specified, prompts sent by the user will be
                            stateless and "one-shot". The LLM will provide a single response to the
                            prompt, without any context from previous prompts.


                            If not specified, "completions" will be used as the default.
                          enum:
                          - chat
                          - completions
                          type: string
                      required:
                      - aiCloudProvider
                      - identifier
                      type: object
                    maxItems: 64
                    minItems: 1
                    type: array
                required:
                - cloudHosted
                type: object
                x-kubernetes-validations:
                - message: At least one class of LLMs has been configured
                  rule: (self.cloudHosted.size() != 0)
            required:
            - gatewayClassName
            type: object
            x-kubernetes-validations:
            - message: At least one type of LLM has been specified
              rule: (self.largeLanguageModels != null)
          status:
            description: Status is the observed state of the AIGateway.
            properties:
              conditions:
                default:
                - lastTransitionTime: "1970-01-01T00:00:00Z"
                  message: Waiting for controller
                  reason: Pending
                  status: Unknown
                  type: Accepted
                description: |-
                  Conditions describe the current conditions of the AIGateway.


                  Known condition types are:


                    - "Accepted"
                    - "Provisioning"
                    - "EndpointsReady"
                items:
                  description: "Condition contains details for one aspect of the current
                    state of this API Resource.\n---\nThis struct is intended for
                    direct use as an array at the field path .status.conditions.  For
                    example,\n\n\n\ttype FooStatus struct{\n\t    // Represents the
                    observations of a foo's current state.\n\t    // Known .status.conditions.type
                    are: \"Available\", \"Progressing\", and \"Degraded\"\n\t    //
                    +patchMergeKey=type\n\t    // +patchStrategy=merge\n\t    // +listType=map\n\t
                    \   // +listMapKey=type\n\t    Conditions []metav1.Condition `json:\"conditions,omitempty\"
                    patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t
                    \   // other fields\n\t}"
                  properties:
                    lastTransitionTime:
                      description: |-
                        lastTransitionTime is the last time the condition transitioned from one status to another.
                        This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
                      format: date-time
                      type: string
                    message:
                      description: |-
                        message is a human readable message indicating details about the transition.
                        This may be an empty string.
                      maxLength: 32768
                      type: string
                    observedGeneration:
                      description: |-
                        observedGeneration represents the .metadata.generation that the condition was set based upon.
                        For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
                        with respect to the current state of the instance.
                      format: int64
                      minimum: 0
                      type: integer
                    reason:
                      description: |-
                        reason contains a programmatic identifier indicating the reason for the condition's last transition.
                        Producers of specific condition types may define expected values and meanings for this field,
                        and whether the values are considered a guaranteed API.
                        The value should be a CamelCase string.
                        This field may not be empty.
                      maxLength: 1024
                      minLength: 1
                      pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
                      type: string
                    status:
                      description: status of the condition, one of True, False, Unknown.
                      enum:
                      - "True"
                      - "False"
                      - Unknown
                      type: string
                    type:
                      description: |-
                        type of condition in CamelCase or in foo.example.com/CamelCase.
                        ---
                        Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be
                        useful (see .node.status.conditions), the ability to deconflict is important.
                        The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
                      maxLength: 316
                      pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
                      type: string
                  required:
                  - lastTransitionTime
                  - message
                  - reason
                  - status
                  - type
                  type: object
                maxItems: 8
                type: array
                x-kubernetes-list-map-keys:
                - type
                x-kubernetes-list-type: map
              endpoints:
                description: |-
                  Endpoints are collections of the URL, credentials and metadata needed in
                  order to access models served by the AIGateway for inference.
                items:
                  description: AIGatewayEndpoint is a network endpoint for accessing
                    an AIGateway.
                  properties:
                    conditions:
                      default:
                      - lastTransitionTime: "1970-01-01T00:00:00Z"
                        message: Waiting for controller
                        reason: Pending
                        status: Unknown
                        type: Provisioning
                      description: |-
                        Conditions describe the current conditions of the AIGatewayEndpoint.


                        Known condition types are:


                          - "Provisioning"
                          - "EndpointReady"
                      items:
                        description: "Condition contains details for one aspect of
                          the current state of this API Resource.\n---\nThis struct
                          is intended for direct use as an array at the field path
                          .status.conditions.  For example,\n\n\n\ttype FooStatus
                          struct{\n\t    // Represents the observations of a foo's
                          current state.\n\t    // Known .status.conditions.type are:
                          \"Available\", \"Progressing\", and \"Degraded\"\n\t    //
                          +patchMergeKey=type\n\t    // +patchStrategy=merge\n\t    //
                          +listType=map\n\t    // +listMapKey=type\n\t    Conditions
                          []metav1.Condition `json:\"conditions,omitempty\" patchStrategy:\"merge\"
                          patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t
                          \   // other fields\n\t}"
                        properties:
                          lastTransitionTime:
                            description: |-
                              lastTransitionTime is the last time the condition transitioned from one status to another.
                              This should be when the underlying condition changed.  If that is not known, then using the time when the API field changed is acceptable.
                            format: date-time
                            type: string
                          message:
                            description: |-
                              message is a human readable message indicating details about the transition.
                              This may be an empty string.
                            maxLength: 32768
                            type: string
                          observedGeneration:
                            description: |-
                              observedGeneration represents the .metadata.generation that the condition was set based upon.
                              For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
                              with respect to the current state of the instance.
                            format: int64
                            minimum: 0
                            type: integer
                          reason:
                            description: |-
                              reason contains a programmatic identifier indicating the reason for the condition's last transition.
                              Producers of specific condition types may define expected values and meanings for this field,
                              and whether the values are considered a guaranteed API.
                              The value should be a CamelCase string.
                              This field may not be empty.
                            maxLength: 1024
                            minLength: 1
                            pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
                            type: string
                          status:
                            description: status of the condition, one of True, False,
                              Unknown.
                            enum:
                            - "True"
                            - "False"
                            - Unknown
                            type: string
                          type:
                            description: |-
                              type of condition in CamelCase or in foo.example.com/CamelCase.
                              ---
                              Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be
                              useful (see .node.status.conditions), the ability to deconflict is important.
                              The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
                            maxLength: 316
                            pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
                            type: string
                        required:
                        - lastTransitionTime
                        - message
                        - reason
                        - status
                        - type
                        type: object
                      maxItems: 8
                      type: array
                      x-kubernetes-list-map-keys:
                      - type
                      x-kubernetes-list-type: map
                    consumer:
                      description: |-
                        Consumer is a reference to the Secret that contains the credentials for
                        the Kong consumer that is allowed to access this endpoint.
                      properties:
                        name:
                          description: Name is the name of the reference object.
                          type: string
                        namespace:
                          description: Namespace is the namespace of the reference
                            object.
                          type: string
                      required:
                      - name
                      - namespace
                      type: object
                    models:
                      description: |-
                        AvailableModels is a list of the identifiers of all the AI models that are
                        accessible from this endpoint.
                      items:
                        type: string
                      type: array
                    network:
                      description: |-
                        NetworkAccessHint is a hint to the user about what kind of network access
                        is expected for the reachability of this endpoint.
                      type: string
                    url:
                      description: |-
                        URL is the URL to access the endpoint from the network indicated by the
                        NetworkAccessHint.
                      type: string
                  required:
                  - consumer
                  - models
                  - network
                  - url
                  type: object
                maxItems: 64
                type: array
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}
